Article may be outdated

This article is 6 days old. Some details may have changed since publication.

Hacker News·5 min read·hard

Can a Linear Program Do Weight-Space Programming?

E
emschwartz
✦AI Summary

Researchers have explored 'weight-space programming,' a technique where AI models like GPT-6 Astra directly edit the weights of other models to improve performance. By treating weight modification as a linear programming problem, researchers successfully reduced reasoning loops in smaller models.

Why it matters

This approach suggests a future where AI can self-optimize or be fine-tuned through direct mathematical intervention rather than traditional training loops, potentially increasing efficiency.

✦Dive DeeperCreate a free account to unlock

Last week, MIT CSAIL published Sparks of Weight-Space Programming . Given the raw weights of a small network, GPT-6 Astra could tell which game it had been trained to play. Given only a game's rules, it could write the weights of a small network from scratch and beat networks trained with PPO for 100 million steps. No gradients, no training loop, just reasoning about what the weights compute. They coined this weight-space programming .

In another experiment, they asked Astra to make Qwen3.5-0.8B better at math by editing its weights. Astra changed only 0.016% of the weights, and MATH-500 accuracy jumped from 7.9% to 48.1%. 1

Continue reading on Headlinne

Create a free account to read the full article.

Read full article →
technologyscience
✦

Get smarter about the news

Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.

Create free account

Already have an account? Sign in