Large Flow Language Models

Arnav Chavan^1,2,* · Chanhyuk Lee^3,* · Udbhav Bamba^2,* · Jaehoon Yoo³ · Seunghoon Hong³ · Aditi Raghunathan^4,† · Jinwoo Kim^3,†

_{¹ UC Berkeley · ² Amazon · ³ KAIST · ⁴ Carnegie Mellon University
^* Equal contribution · ^† Equal advising}

---

Lafa is a continuous flow language model that generates text block by block through a flow over token embeddings, which can be distilled into a flow map for one- or few-step generation. Lafa can also verify its own flow drafts autoregressively in self-speculative decoding, reaching 2.54× the throughput of an autoregressive model at comparable quality.

Results

Accuracy vs. throughput of Lafa 7B against autoregressive, multi-token prediction, and diffusion baselines, averaged over IFEval, GSM8K, and HumanEval. Lafa 7B reaches 2.54× the throughput of an autoregressive model at matched or better quality.

Models

All released models use block size 4. Each checkpoint supports flow, autoregressive, and self-speculative decoding.

Backbone	Size	Block	Model
Qwen2.5 Instruct	0.5B	4	🤗 lafa-hf/lafa-0.5B
Qwen2.5 Instruct	7B	4	🤗 lafa-hf/lafa-7B
Qwen3 Base	8B	4	🤗 lafa-hf/lafa-8B

Quick start

Install the inference dependencies, then load a model from Hugging Face. The model code ships with the checkpoint.

git clone https://github.com/lafa-gh/large-flow-lms.git
cd large-flow-lms
pip install -r requirements-inference.txt

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "lafa-hf/lafa-0.5B"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    trust_remote_code=True,
    dtype=torch.bfloat16,
).to("cuda").eval()

text = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is 17 + 25?"}],
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(
    text,
    add_special_tokens=False,
    return_tensors="pt",
).to("cuda")

output = model.generate(
    **inputs,
    gen_mode="flow",
    num_steps=4,
    noise_scale_override=0.1,
    cfg_scale=1.0,
    max_new_tokens=128,
)

print(tokenizer.decode(
    output[0, inputs.input_ids.shape[1]:],
    skip_special_tokens=True,
))

Change the arguments to switch decoding:

Decoding	Arguments
Flow	gen_mode="flow", num_steps=K
Autoregressive	gen_mode="ar"
Self-speculative	gen_mode="speculative"
Confidence thresholding	gen_mode="flow", num_steps=1, threshold=0.9, self_cond=False

Guides

Detailed instructions for each part of the pipeline:

Inference	Modes, batching, and checkpoint options
Training	Convert or continue training a Lafa model
Distillation	Train a few-step flow map
Evaluation	Run benchmarks and checkpoint checks

Citation

If you use Lafa, please cite:

@article{chavan2026largeflow,
  title   = {Large Flow Language Models},
  author  = {Chavan, Arnav and Lee, Chanhyuk and Bamba, Udbhav and Yoo, Jaehoon
             and Hong, Seunghoon and Raghunathan, Aditi and Kim, Jinwoo},
  journal = {arXiv preprint arXiv:2610.28559},
  year    = {2026}
}