Mechanistic Interpretability is a 2026 Breakthrough Technology. Here's What That Means for the "LLMs Are Just Matrix Multiplication" Debate

Today a friend of mine — let's leave him nameless — said the line I've been hearing since 2022: "It's still just matrices multiplying, guessing the most probable next word." For a long while I had no good rebuttal beyond intuition. I've been using LLMs as a daily tool since the original ChatGPT ship