https://tomarcher.io/learningpaths/how-large-language-models-work/Recent content in How Large Language Models Work on Signal & SyntaxHugoen-usSat, 22 Aug 2026 06:00:00 -0700https://tomarcher.io/posts/inside-attention-part-1/Sat, 22 Aug 2026 06:00:00 -0700https://tomarcher.io/posts/inside-attention-part-1/The transformer architecture is composed of many repeating transformer layers, or blocks. Each block contains an attention sublayer followed by a feedforward sublayer, wrapped in residual connections and layer normalization. Positional information is added to the input so the model knows what order the tokens came in. The attention sublayer sets the table for the feedforward sublayer: it does the work of looking at other tokens and deciding what information to absorb from them.https://tomarcher.io/posts/minimal-transformer/Wed, 01 Apr 2026 06:00:00 -0700https://tomarcher.io/posts/minimal-transformer/In 2017, Vaswani et al. published “Attention Is All You Need,” a paper that quietly rearranged the entire landscape of machine learning. It introduced the Transformer architecture — a design that has since become the backbone of every major language model you’ve heard of: GPT, BERT, Claude, Gemini, and dozens of others. The paper’s title was a provocation. Attention mechanisms already existed. The claim was that you could throw out recurrence entirely and let attention carry the whole load.https://tomarcher.io/posts/discrete-math-in-large-language-models/Tue, 31 Mar 2026 06:00:00 -0700https://tomarcher.io/posts/discrete-math-in-large-language-models/A recent LinkedIn post from Michael Palmer described how discrete mathematics is the foundation for how computers reason about problems. That thread got me thinking about just how many discrete math concepts show up inside systems that seem purely statistical. LLMs are often described in terms of neural networks, gradient descent, and probability distributions. If you’ve taken discrete mathematics and wondered what it has to do with modern AI, the answer is: more than you’d expect.https://tomarcher.io/posts/how-large-language-models-know-things-they-were-never-taught/Mon, 09 Feb 2026 06:00:00 -0700https://tomarcher.io/posts/how-large-language-models-know-things-they-were-never-taught/When you ask an LLM without web search enabled a question like “What happened in the news this morning?”, the LLM will respond by telling you that it doesn’t have access to current events and suggest you check a more current news source such as Reuters or Google News. Conversely, ask an LLM with web search enabled the same question, and you receive a detailed rundown of breaking stories, political controversies, and sports news from the past 24 hours.https://tomarcher.io/posts/temperature-top-p-creativity-knobs/Wed, 24 Dec 2025 12:00:00 -0800https://tomarcher.io/posts/temperature-top-p-creativity-knobs/Every API call to ChatGPT , Claude , or any other LLM includes two parameters most people either ignore or tweak randomly: temperature and top-p. The defaults work fine for casual use, so why bother understanding them? Because these two numbers fundamentally control how your model thinks. The temperature value determines whether the model plays it safe or takes creative risks while the top-p value decides how many options the model even considers.https://tomarcher.io/posts/how-large-language-models-tokenize-text/Tue, 11 Nov 2025 06:00:00 -0700https://tomarcher.io/posts/how-large-language-models-tokenize-text/When you type “I love programming” into ChatGPT, you might assume the model reads three words. It doesn’t. It reads somewhere between three and seven tokens, depending on how the text is split. When you ask Claude to count the letters in the word “strawberry,” it often gets it wrong. The reason is simple. Claude never saw the word “strawberry” as a complete unit. It saw tokens like "str", "aw", "berry" and tried to reason about letters it couldn’t directly access.https://tomarcher.io/posts/how-large-language-models-handle-context-windows/Mon, 10 Nov 2025 06:00:00 -0700https://tomarcher.io/posts/how-large-language-models-handle-context-windows/When you have a long conversation with a large language model (LLM) such as ChatGPT or Claude , it feels like the model remembers everything you’ve discussed. It references earlier points, maintains consistent context, and seems to “know” what you talked about pages ago. But here’s the uncomfortable truth: the model doesn’t remember anything. It’s not storing your conversation in memory the way a database would. Instead, it’s rereading the entire conversation from the beginning every single time you send a message.https://tomarcher.io/posts/how-large-language-models-learn/Wed, 08 Oct 2025 06:00:00 -0700https://tomarcher.io/posts/how-large-language-models-learn/When you interact with a large language model (LLM) such as ChatGPT or Claude , the model seems to respond instantly relative to the question’s degree of difficulty. What’s easy to forget is that every word it predicts comes from a long history of learning where billions of gradient steps have slowly sculpted its understanding of language. Large language models don’t memorize text. They optimize it. Behind that optimization lies calculus.https://tomarcher.io/posts/how-large-language-models-think/Tue, 07 Oct 2025 06:00:00 -0700https://tomarcher.io/posts/how-large-language-models-think/When you enter a sentence into a Large Language Model (LLM) such as ChatGPT or Claude , the model does not process words as language. It represents them as numbers. Each word, phrase, and code token becomes a vector — a list of real-valued coordinates within a high-dimensional space. Relationships between meanings are captured not by grammar or logic but by geometry. The closer two vectors lie, the more similar their semantic roles appear to the model.