What is semi-structured data?
Semi-structured data carries its own labels but does not conform to a rigid table. A JSON document from a web service, an Extensible Markup Language (XML) invoice exchanged over electronic data interchange (EDI), a server log line, and an email header all have named parts that a program can read, yet two records in the same feed may have different fields, nested lists, or missing values.
The structure lives inside each record rather than in a schema declared ahead of time. That makes the format flexible for the sender and more work for the receiver, who has to handle every variation that shows up.
Semi-structured data in practice
A freight forwarder receives shipment status events from dozens of carriers. Each carrier sends JSON, but one nests the container number under an equipment object, another puts it at the top level, and a third omits it for less-than-truckload moves. The forwarder’s data pipeline parses each variant, maps it to a canonical schema, and loads the result into a lakehouse table where ordinary database queries work again. Data platforms now store raw messages as they arrive and apply the schema on read, so a new carrier format does not break the nightly load.
Large language models help here. Given a sample record, a model can draft the mapping code and flag the fields it could not place. API responses, event streams, and tool outputs in agent systems are nearly all semi-structured, so handling this kind of data well is a prerequisite for connecting agents to business systems.