The Download: a “dire wolf” revival, and safeguarding AI companions

“No longer do I have to drive a symbol of racism, greed and ignorance! Life is suddenly so much better!”

—Actor Bette Middler expresses her joy at selling her Tesla, Insider reports.

The big story

Large language models can do jaw-dropping things. But nobody knows exactly why.

Two years ago, Yuri Burda and Harri Edwards, researchers at OpenAI, were trying to find out what it would take to get a large language model to do basic arithmetic. At first, things didn’t go too well. The models memorized the sums they saw but failed to solve new ones.

By accident, Burda and Edwards left some of their experiments running for days rather than hours. The models were shown the example sums over and over again, and eventually they learned to add two numbers—it had just taken a lot more time than anybody thought it should.

In certain cases, models could seemingly fail to learn a task and then all of a sudden just get it, as if a lightbulb had switched on, a behavior the researchers called grokking. Grokking is just one of several odd phenomena that have AI researchers scratching their heads. The largest models, and large language models in particular, seem to behave in ways textbook math says they shouldn’t.

This highlights a remarkable fact about deep learning, the fundamental technology behind today’s AI boom: for all its runaway success, nobody knows exactly how—or why—it works. Read the full story.

—Will Douglas Heaven

We can still have nice things

A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line or skeet ’em at me.)

+ What happened when Wham! took Western pop music to China 40 years ago.
+ Who knew that sharks do make noises after all? 🦈
+ Microsoft is celebrating its 50th anniversary, and with it, 50 seriously strange inventions.
+ What the heck is a prototaxite, anyway?