What is topic modeling?
Topic modeling is an unsupervised learning technique that discovers the themes running through a large collection of documents without being told in advance what to look for. The algorithm groups words that tend to appear together and represents each document as a mixture of those groups. A classic method, latent Dirichlet allocation (LDA), produces topics as lists of weighted words that an analyst then names. Newer approaches cluster document embeddings and ask a large language model to label each cluster in plain language.
Topic modeling differs from text classification in that the categories are not defined beforehand. It is a discovery tool for learning what people are talking about before deciding which categories matter.
Topic modeling in practice
A hospital system analyzes two years of patient complaint letters. Topic modeling surfaces clusters around discharge paperwork, parking, billing confusion, and communication with night staff, including one about a specific wing that nobody had tracked. The patient experience team uses those clusters to define categories for an ongoing classifier and to prioritize fixes. The same technique helps a manufacturer read thousands of warranty claim narratives and spot a failure mode early.
What to watch for
- Expect to iterate on the number of topics; too few merge unrelated themes and too many split one theme into pieces.
- Have someone who knows the business name the topics, since the raw word lists are often ambiguous.
- Treat the output as a starting point for a supervised classifier rather than as a finished reporting category.