5 "Probability" Posts

Temperature and Top-P: The Creativity Knobs

How sampling parameters shape AI personality
What You'll Learn
  • How an LLM turns raw token scores into probabilities before choosing what comes next
  • How temperature reshapes a probability distribution to make outputs more predictable or more varied
  • How top-p sampling limits which tokens the model is allowed to consider
  • Why top-p adapts to model confidence differently from a fixed top-k cutoff
  • How temperature and top-p interact when both are applied to the same distribution
  • How to choose sampling settings for factual, structured, professional, and creative tasks

The Birthday Paradox in Production: When Random IDs Collide

Why collision risk grows faster than intuition suggests, and what that means for IDs, hashes, and distributed systems
What You'll Learn
  • Why collisions happen long before an ID space is full
  • Why collision risk grows much faster than intuition suggests
  • How the birthday paradox applies to computer systems
  • How to calculate the probability of an ID collision
  • Why the square root of the ID space determines the danger zone
  • How 32-bit, 64-bit, and UUID v4 IDs compare
  • How generation rate changes the time until collisions become likely
  • How to choose an acceptable collision risk for a real system
  • How Monte Carlo simulation can validate collision calculations
  • How to predict when an ID strategy needs to be replaced

How Large Language Models (LLMs) Read Code: Seeing Patterns Instead of Logic

Exploring how large language models interpret code and what they miss
What You'll Learn
  • How an LLM reads code differently from a compiler or a human developer
  • Why models recognize programming patterns instead of executing the code they see
  • How embeddings let an LLM associate code with similar structures and meanings
  • Why comments, variable names, and familiar coding idioms can change a model’s interpretation
  • How statistically likely code can still be logically or operationally wrong
  • Why combining generative AI with compilers and static analysis produces safer coding tools