What is data integration design?
Data integration design is the detailed specification for one flow of data between systems. Data integration architecture sets the company-wide patterns, and the design applies them to a single connection, such as online orders flowing into the ERP. A complete design names the source and target, maps each field, and documents every transformation, such as unit conversions and code translations. It also states how records are matched, what schedule or event starts the flow, and whether a record that fails data validation is rejected, held for review, or loaded with a flag.
Data integration design in practice
A third-party logistics provider onboards a new client whose orders arrive as a nightly file. The design maps the client’s item numbers to the warehouse’s own, converts cases to single units, and translates carrier codes into the ones the warehouse management system uses. Orders for unknown items go to an exception queue that the account manager clears before picking starts. The same document doubles as the test plan, since every mapping rule is a case to verify. A language model can draft first-pass mappings from sample records, but a person who knows both systems should approve them.
What to watch for
- Record the business meaning of each mapped field, because two systems can use one field name for different things.
- Design the error path as carefully as the normal path, since unhandled failures are where records go missing.
- Reconcile record counts or totals between source and target after each run.