What is unstructured data?
Unstructured data is information that does not fit a predefined set of fields. Contracts, emails, scanned invoices, meeting recordings, support chat logs, photographs, and engineering drawings are all unstructured. The content has meaning to a person who reads or views it, but a database cannot query it the way it queries a column of order dates.
Much of the information a company holds is unstructured, and it accumulates faster than the structured records do. For years the only practical way to use it was to have people read it and type the relevant facts into a structured system, which is what a claims processor or an accounts payable clerk spends much of the day doing.
Why unstructured data matters
Generative AI changed the economics of unstructured data. Large language models read text, vision-language models read images and scanned pages, and speech recognition turns audio into transcripts. A model can pull the renewal date and the liability cap out of a 40-page contract, or classify an inbound email by intent, without anyone writing field-by-field extraction rules. Retrieval-augmented generation (RAG) goes a step further and lets staff ask questions across thousands of documents at once.
The practical result is that documents, which used to be an archive, become a data source. Intelligent document processing turns them into structured records. Embeddings and a vector database make them searchable by meaning rather than by keyword.
Unstructured data in practice
A regional law firm holds twenty years of engagement letters, pleadings, and correspondence in a document management system that associates search by file name and client. After the firm indexes the corpus with embeddings, an attorney can ask for every matter where a particular indemnification clause was negotiated and get the passages, with citations, in seconds. The same index later feeds a conflict-check agent that flags matters involving a prospective client’s affiliates.
What to watch for
- Permissions travel poorly. A document that only partners could open must stay restricted when its text lands in a search index, which is what permission-aware retrieval addresses.
- Scanned documents need optical character recognition (OCR) first, and poor scans produce poor text no matter how capable the model is.
- Unstructured data is where personally identifiable information hides, in free-text notes and attachments, so classification and redaction belong in the pipeline.
- Extraction from documents is probabilistic. Keep a confidence score on each extracted field and route low-confidence results to a person.
- Sending a company’s contracts and emails to a rented model raises residency and confidentiality questions that structured aggregates rarely do, and a private deployment keeps the corpus inside the firm’s own infrastructure.