What is a neural network?
A neural network, more fully an artificial neural network, is a machine learning model built from layers of simple units that pass numbers to one another. Each unit receives inputs, multiplies each by a weight, adds them up, and passes the sum through an activation function that decides how strongly to fire. Thousands or billions of these units, arranged in layers, can approximate almost any relationship between inputs and outputs.
The name comes from a loose analogy with neurons in the brain, but the resemblance ends there. A neural network is arithmetic: matrix multiplications followed by simple nonlinear functions, repeated layer by layer. Its knowledge is stored entirely in the weights, which is why a trained model is distributed as a file of weights.
How a neural network works
Data enters at the input layer, one unit per feature, such as one unit per pixel of an image or per field of a record. It flows through one or more hidden layers and emerges at the output layer, which might be a single number for a forecast or one unit per category for a classification. This forward pass is how the network makes a prediction.
Learning happens by adjusting the weights. During training, the network’s output is compared with the correct answer using a loss function. Backpropagation then works backward through the layers to calculate how much each weight contributed to the error, and gradient descent moves each weight a small step in the direction that reduces it. The step size is the learning rate. After many passes over the training data, called epochs, the weights encode the pattern.
The activation function is what gives the network its flexibility. Without it, stacking layers would be no better than a single linear equation. Common choices include the rectified linear unit (ReLU), which passes positive values and zeroes out negative ones, and the sigmoid, which squashes any value into a range between zero and one.
Types of neural network
- Feedforward networks (multilayer perceptrons) pass data straight through fully connected layers and suit tabular data and simple scoring tasks.
- Convolutional neural networks scan images with small filters and dominate visual inspection and image recognition.
- Recurrent neural networks, including the LSTM, process sequences one step at a time and were long the standard for speech and time series.
- Transformers use attention to relate every element of a sequence to every other, and underlie large language models.
- Autoencoders learn to compress and reconstruct their input and are used for anomaly detection and dimensionality reduction.
Neural networks in practice
A regional trucking company wants better delivery time estimates than its dispatchers’ rules of thumb. A feedforward network takes distance, route segments, time of day, day of week, weather, trailer type, and the customer’s historical dock wait as inputs and outputs an estimated arrival time. Trained on two years of completed loads, it is judged by mean absolute error on recent weeks that were held out. Dispatchers see the estimate alongside the model’s own confidence and override it when they know something the data does not.
What to watch for
- A neural network needs more data than a classic model to beat it, so for small tabular datasets try gradient-boosted trees first.
- Inputs must be normalized to similar ranges, or training becomes slow and unstable.
- Networks memorize when they are too large for the data. Use validation data to detect overfitting and regularization to limit it.
- The weights are not readable by people. Budget for explanation tools such as Shapley values where decisions must be justified.