LoRA Speedrun – a public wall-clock leaderboard for fine-tuning techniques
The LoRA Speedrun is a public leaderboard designed to standardize and benchmark parameter-efficient fine-tuning techniques for large language models. By using a frozen task, hardware, and dataset, it allows researchers to compare the efficiency of various LoRA-based methods in a controlled environment.
Why it matters
It provides a much-needed apples-to-apples comparison for AI fine-tuning research, helping the community identify the most efficient methods for training models on limited hardware.
How fast can you LoRA-fine-tune Qwen2.5-1.5B to ≥ 57% on GSM8K — on a single L40S?
This is modded-nanogpt for fine-tuning: a frozen task, frozen hardware, and a public leaderboard of wall-clock records. Every record is independently re-run 3× with fresh seeds on identical hardware before it counts.
Attempting and verifying are free : official timing runs on a Modal L40S sandbox, and Modal's free monthly compute credits cover full runs — so anyone can compete, and anyone can re-verify any record with one command.
Current record: 6m 05s by @Saivineeth147 — Sequence packing + completion-only loss masking, 2 epochs. Same LoRA config as #0; ~2x faster at higher accuracy.
Full history with verification reports: records/RECORDS.md
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in