Can a Linear Program Do Weight-Space Programming?
Researchers have explored 'weight-space programming,' a technique where AI models like GPT-6 Astra directly edit the weights of other models to improve performance. By treating weight modification as a linear programming problem, researchers successfully reduced reasoning loops in smaller models.
Why it matters
This approach suggests a future where AI can self-optimize or be fine-tuned through direct mathematical intervention rather than traditional training loops, potentially increasing efficiency.
Last week, MIT CSAIL published Sparks of Weight-Space Programming . Given the raw weights of a small network, GPT-6 Astra could tell which game it had been trained to play. Given only a game's rules, it could write the weights of a small network from scratch and beat networks trained with PPO for 100 million steps. No gradients, no training loop, just reasoning about what the weights compute. They coined this weight-space programming .
In another experiment, they asked Astra to make Qwen3.5-0.8B better at math by editing its weights. Astra changed only 0.016% of the weights, and MATH-500 accuracy jumped from 7.9% to 48.1%. 1
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in