Teaching Small Language Models to Remember: Giving LLMs a Notebook with Differentiable Neural Computers

"Large models memorize the world in their weights. Small models need a notepad." Large Language Models (LLMs) like GPT-4 are remarkably good at recalling facts — "Delhi is the capital of India," "Einstein developed the theory of relativity" — because they have billions of parameters acting as a mass