What is self-supervised learning?
Self-supervised learning is machine learning in which the training signal comes from the data itself rather than from human-written labels. Part of each example is hidden, and the model is trained to reconstruct it. Because the hidden part is already known, every document, image, or recording becomes a labeled example at no labeling cost.
For text, the usual task is predicting the next token given the ones before it, or filling in masked words, as BERT does. For images, one version hides patches of the picture, and another asks the model to recognize two altered views of the same photo as a match. In every case the model is forced to learn structure, grammar, and meaning in order to guess well.
Why self-supervised learning matters
This is how foundation models are pre-trained. A large language model reads an enormous quantity of text with no labels at all, learning language and a great deal of world knowledge along the way. The pre-trained model is then adapted with a comparatively small amount of supervised or human-feedback data, through instruction tuning and reinforcement learning from human feedback, so that it follows instructions.
For a business, the consequence is that general capability arrives in the model already. Fine-tuning on a few thousand company documents, or transfer learning from a pre-trained vision model, costs far less than training from scratch. This is what makes private AI deployments on a company’s own infrastructure practical: the expensive learning has already been done.