What is a deep neural network?
A deep neural network (DNN) is a neural network with multiple hidden layers between its input and its output. Early networks had a single hidden layer and could only learn simple patterns. Adding depth lets each layer build on the features the previous one found, so the network learns a hierarchy: raw pixels, then edges, then textures, then parts, then whole objects. Deep learning is the practice of training such networks.
There is no fixed threshold for “deep.” Networks in production range from three or four hidden layers for tabular scoring to hundreds of layers in image models and large language models.
How a deep neural network works
Training uses the same backpropagation and gradient descent as any neural network, but depth brings problems. Error signals shrink as they pass backward through many layers, a failure called the vanishing gradient, which once made deep networks impossible to train. Three changes fixed it: the ReLU activation function, which does not squash gradients; batch normalization, which keeps each layer’s values in a stable range; and residual connections, which let a signal skip layers. Graphics processing units supplied the compute, and large labeled datasets such as ImageNet supplied the examples. At inference time a deep network is a long chain of matrix multiplications, so its speed depends on parameter count and hardware.
Deep neural networks in practice
A payments processor screens card transactions for fraud in real time. A deep network takes hundreds of inputs per transaction, including amount, merchant category, device fingerprint, and the cardholder’s recent pattern, and returns a fraud probability within a few milliseconds. The depth lets it learn interactions between features that no analyst would think to code. Transactions above a threshold are held for review, and the threshold is tuned against the cost of blocking a good customer. The same family, with convolutional or attention layers added, runs speech recognition in call centers and document reading in insurance claims.
What to watch for
- Deeper is not automatically better. Extra layers add training cost and overfitting risk without improving accuracy once the data is exhausted.
- Deep networks are opaque, so decide before building how a flagged transaction or a rejected application will be explained.
- Measure inference latency and memory on the target hardware, since a model that fits a training server may not fit an edge device.
- Start from a pre-trained network where one exists for your data type and fine-tune it rather than training from zero.