On Scaling Limits and Open Source
Tsiolkovsky, Shannon, and OpenMythos
In 1897, a Russian schoolteacher working alone in Kaluga wrote down a relationship between fuel and speed that has governed every rocket ever launched. He was not trying to build anything. He was trying to understand whether a rocket was even possible in the vacuum of space, where there is nothing to push against. What he found was not a solution. It was a wall.
The wall says: the faster you want to go, the more fuel you need — but not proportionally more. Exponentially more. Each additional unit of speed costs a multiplied fraction of everything you are already carrying. You end up burning fuel to carry the fuel that carries the fuel. To reach orbit, fifteen kilograms of propellant burn for every single kilogram that arrives. The rocket equation does not negotiate. It does not respond to better engineering. It is a mathematical sentence, and the only real answer ever found was staging — shed the spent mass at each phase, apply the cost locally, reset the calculation. The wall holds. But you stop paying the whole bill at once.
Tsiolkovsky’s wall and Claude Shannon’s ceiling are the same wall. Shannon proved it in 1948 from a completely different direction: every channel that carries information — a wire, a radio signal, a conversation — has a hard limit on how much it can reliably transmit. Past that limit, the channel does not go silent. It does not produce recognizable static. It produces output that looks exactly like signal — structured, coherent, formatted correctly — but carries nothing. The noise has learned to dress as a transmission. An observer on the receiving end has no local way to tell the difference.
Shannon called this saturation failure. He was precise: it is not an engineering limitation to be eventually overcome. It is a boundary condition baked into the physics of the channel itself.
This is what AI hallucination actually is. Not a fine-tuning problem. Not a prompt engineering problem. A language model is a channel. When a question exceeds what the architecture can reliably carry — when the correct answer lives on the other side of the ceiling — the model does not go quiet. It produces fluent, confident, syntactically perfect output that may have no relationship to the truth. A saturated channel emitting noise it has learned to dress as signal. Making the model larger raises the ceiling slightly. It does not change the nature of the ceiling. The hardest questions expand to fill whatever ceiling you build.
The Confession the Numbers Are Making
The AI industry has been slowly, expensively arriving at this reckoning. The jump from GPT-2 to GPT-4 required roughly ten thousand times more compute. The capability improvement was not ten thousand times larger. Each new order of magnitude of investment buys less and less on the problems that actually require deep reasoning — chained inference, long-context judgment, the kind of thinking that holds many facts in mind simultaneously and draws conclusions that do not live on the surface. These are not harder in degree from the easy questions. They are harder in kind. They sit on the other side of a qualitative boundary, and the strategy of making the same architecture larger and training it on more data does not cross a qualitative boundary. It approaches it. More expensively with each iteration.
This is Tsiolkovsky in a data center. The rocket is getting heavier. The staging has not yet happened.
Claude Mythos — Anthropic’s frontier reasoning model, the architecture that has been held up as the apex of what deep reasoning in a language model can look like — cannot scale its way out of this. The compute cost of pushing further along the same architectural logic is becoming structurally untenable. The ceiling is real. The exponential is real. More fuel is not the answer.
Parcae, and Time Dilation at the Frontier
Two weeks ago, researchers at UC San Diego and Together AI published an architecture called Parcae. It is a looped model: rather than processing a question once through a fixed sequence of stages, it loops through the same stages repeatedly, refining its understanding with each pass. Easy questions exit early. Hard questions loop longer. For the first time, a model allocates effort as a function of actual difficulty — not distributing compute uniformly across easy and hard alike, but going deeper where depth is warranted and stopping when the answer has crystallized.
The stability problem in looped architectures has always been drift: accumulated reasoning that amplifies error rather than insight, loop after loop, until the output collapses into noise. Parcae solves this not by monitoring for drift but by making drift structurally impossible. The loop is constrained to be contractive by construction. It must converge. A 770 million parameter Parcae model matches a 1.3 billion parameter standard model in quality. Half the parameters. Clean, predictable scaling laws that earlier looped architectures could never produce. The staging, in the Tsiolkovsky sense, has happened.
Parcae is the staging mechanism. OpenMythos is the moment it became public. The frontier labs are the single-stage rocket that just watched the staging happen on GitHub.