Modern LLMs: What the FLIP is an Engram!?

Modern LLMs are built out of two primary components. Some kind of perceptron, like a MoE or a dense layer, and attention. Attention mixes in sequence information, and perceptrons do everything else. As you can probably imagine, that means perceptrons have to learn a lot of things and many of those things are quite hard. Engrams pick up the slack on some of these things, allowing the perceptrons to devote more of their parameters to the abstract witchcraft letting them capture lingustic concepts. Engrams get billed as memory but I think that's too mysterious. It's a specific kind of memory they're actually able to capture.

Lets first detour into why anyone would call an engram memory. Engrams are embeddings, literally. So, lets talk about the simpler thing, embeddings. Are embeddings memory? Sort of. If we have a token 'Dog' the embedding that token id gives represents a dog ~ you might say it's the memory of a dog. Or the concept of a dog. It's kind of abstract, and this is what we're talking about, so if I throw the word memory around, it's important to realize we're really talking about frozen learned concepts and not fungible, articulate memory. Conditional Memory is what Deep Seek calls this, it's a good enough name, probably.

Okay, I'll repeat it again, Engrams are embeddings. With extra steps. Lets consider the path a single word takes to get into the latent space of a normal language model. First, we tokenize it. Lets say the word is "Demonize". My specific tokenizer produces the tokens: "De", "mon", "ize". Each of these tokens has it's own embedding which means the concept of demonize is spread across three different unrelated token embedings. Visually:

The important bit here is that these three tokens are different. They're also unrelated. Now, maybe you're wondering 'How do models actually associate these three tokens with the concept of Demonize?' Great question, and the answer is that attention mixes in the sequence information and the perceptron has to just figure it out. Yes, the perceptron literally has to somehow learn the association of this conditional local sequence. That's pretty hard if we extend out that it has to learn this over all local sequences and also it's concerned with everything else, that's a lot of stuff. Engrams, primarily, alleviate the local sequence information issue and they do that by adding more embeddings. Instead of embedding one token id, we instead embed a hashed-sequence of some number of tokens. So, if we want a bigram, two tokens, we'd hash "De", "mon". Trigram, we'd hash "De", "mon", "ize".

This is essentially all engrams do, and we do this for every position in the context ~ so if we have the sequence "The cat ran fast" we could imagine the first trigram might be "The cat ran" and the second would be "cat ran fast".

In basic terms engrams are literally this simple, it's just like our embedding table, but bigger. However, there's some caveats. Typical embedding tables are equal to the vocabulary size. That is, if my tokenizer has 8,000 token vocab, then my normal embedding table is 8k rows. However, if we're doing bigrams for our engram table, you can see we have two tokens, each drawn from 8k. So we have 8k^2 possible rows, otherwise known as 64 million rows needed for every possible bigram. And that's a tiny vocabulary many modern models have 100k+ sized vocabs. And that's just bigrams! What about trigrams? Or quadgrams, or anything else? How could we possibly support that many rows? And the answer is, you can't really. So, we pick an arbitrary size for the table and accept that the hashes will collide. There's some other strategies typically employed as well, such as using more than one table and using a different hash per table. This means though each table is effectively guaranteed to collide, the aggregate information has high likelihood of being unique. Additionally, modern engram methods proposed by e.g. deep seek or meituan advise using a hierarchy of bigram + trigrams. Or even larger, but recognizing that as you add more combinations, the odds of seeing that sequence rapidly decline under normal language conditions (zipf distributed.)

Because rare sequences are rare (duh) this can become learning-starvation for some concepts of the embedding table, if you only see one or no examples of a sequence, it simply cannot be learned by the model, so if it's encountered in real language the model will likely become confused-by the engram. Remember, the concepts will collide so it's very likely something will fill the row, but maybe not the thing you're currently looking at. To deal with this, the engram typically has a confidence-gate which is intended to deal with situations where the model doesn't think this specific row is useful. Lets make this concept of gating concrete, say we have a trigram and the specific one we encountered is "Doomed Buffalo Explosion." Encountering this specific sequence in text is almost zero, even as a trigram. This means we'd expect, if we ever have this information appear in context, that the engram would give us a very low context-confidence and gate itself out. Conversly, something very common like "She looked at" would likely have extreme representation, and probably dominate it's own row.

In this way, engrams end up acting as a memory for typical sequences and concepts and the model will still have to rely on attention and perceptrons for rarer, unusual, or out of distribution things. Engrams are embeddings. Just instead of one token, they embed a hashed group of multiple tokens.

There's more to the story on a technical level, of course, but the main concept is actually very simple as you see. I'd recommend these excellent papers for more on the subject:

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models Scaling Embeddings Outperforms Scaling Experts in Language Models