Why AI Tokens are so Expensive - Computerphile
The high cost of AI, particularly in agentic coding models, is driven by the complexity and volume of tokens—units of text like words or characters—used in large language models, which require extensive pre-training and refinement across various domains.
MAIN POINTS FROM TRANSCRIPT
- AI costs are rising due to the complexity of token usage in language models.
- Tokens include words, characters, and spaces, with frequency influencing their selection.
- Tokenizers convert text into numerical embeddings for model processing.
- Large models are pre-trained on diverse data before domain-specific refinement.
TAKEAWAYS
- AI models use a vast number of tokens, including common words and unique symbols.
- Tokenization is a critical system development issue in AI model training.
- Pre-training on diverse data enhances model versatility across different tasks.
- The shift to token-based pricing reflects the operational costs of AI models.