Language Models

A language model is a probabilistic model of text. Large language models (LLMs) are neural networks trained on vast corpora of human-written text, learning to predict the next token given prior tokens.

How they work

LLMs use the transformer architecture, introduced in “Attention Is All You Need” (Vaswani et al., 2017). Self-attention allows the model to weigh the relevance of each token in context.

Training involves predicting masked or future tokens across billions of examples. The result is a model that encodes a compressed representation of human knowledge and language patterns.

Emergent capabilities

At sufficient scale, LLMs exhibit capabilities not explicitly trained for: multi-step reasoning, code generation, translation, and apparent common sense. Whether these reflect “understanding” or sophisticated pattern matching is debated.

Philosophical implications

LLMs raise hard questions:

Do they have beliefs? Intentions?
Is their output a form of thought or mimicry?
How does their deterministic computation relate to Free Will?

These questions connect to longstanding debates in Philosophy about mind, consciousness, and agency.

The web connection

LLMs were largely trained on web content — the accumulated written output of humanity. In a sense, they are distillations of the web.