Tagged
#scaling
5 articles
Why context length is hard to scale
A longer context window sounds like a simple knob to turn. Underneath it fights a cost that grows faster than the text — and attention that spreads thin.
What model "parameters" actually are
"Billions of parameters" gets quoted like horsepower. Here is what a parameter really is, why the count matters, and why bigger isn't automatically better.
The economics of inference: why "cheap AI" still adds up
A single AI call looks almost free. So why do AI bills balloon? A plain-language tour of the economics that turn pennies into real money.
Mixture-of-experts models, explained simply
Mixture-of-experts lets a model be huge yet cheap to run by using only a slice of itself per input. Here is the idea, plainly, and why it matters.
Emergent abilities: real or mirage?
Big models seem to suddenly "get" skills smaller ones lack. Is that a real phase change, or a trick of how we measure? The honest answer is: both.