token

A token is how AI models see text. Rather than reading word by word, models break text into tokens — fragments that are roughly 3/4 of a word on average. 'Unbelievable' might be three tokens: 'Un', 'believ', 'able'. Common words like 'the' are usually one token. This matters because everything in AI is priced and limited by tokens.

Context windows are measured in tokens (Claude has 200K, GPT-4 has 128K). API pricing is per-token. Response speed depends on tokens generated. When someone says 'I ran out of context,' they mean they hit the token limit.

Understanding tokens gives you practical superpowers: you can estimate costs, know when to summarize instead of dumping full documents, and understand why some prompts work better than others (shorter prompts leave more room for the response).

When to Use It

When discussing AI costs, context limits, prompt optimization, or debugging why a long conversation started losing quality.

Try This Prompt

Why It Matters

Token literacy is financial literacy for AI. Every wasted token costs money and displaces useful context.

Memory Trick

Think of arcade tokens — you have a limited number, each one lets you play a little more, and when they're gone, the game's over.

Example Prompts

Common Misuses

• ×Thinking 1 token = 1 word (it's roughly 0.75 words)
• ×Confusing tokens with characters — they're different units entirely
• ×Not realizing that both input AND output count toward token limits and costs

Related Power Words