What is a recurrent neural network?
A recurrent neural network (RNN) is a neural network built for sequences such as sentences, sensor readings, or daily sales. It reads the sequence one element at a time and keeps a hidden state, a set of numbers that summarizes everything seen so far. At each step the new input and the previous state combine to produce an output and an updated state. The same weights are applied at every step, so the network can handle sequences of any length.
This loop is what makes the network “recurrent.” It gives the model a form of memory, letting the meaning of a word depend on the words before it, or the significance of a temperature reading depend on the trend of the last hour.
How a recurrent neural network works
Training unrolls the loop into a long chain, one copy of the network per time step, and applies backpropagation through time. Gradients passing back through many steps tend to shrink toward zero or grow without bound, which makes plain RNNs forget anything more than a few dozen steps old. Gated variants fix this: the long short-term memory (LSTM) network and the gated recurrent unit (GRU) add learned gates that decide what to keep and what to discard. For language tasks, transformers have since replaced recurrent networks almost entirely, because they read a whole sequence at once and train in parallel.
Recurrent neural networks in practice
An electric utility forecasts hourly load for each substation a day ahead. An LSTM reads the last two weeks of hourly demand, temperature, and calendar flags and outputs the next twenty-four hours. Recurrent models also remain common in predictive maintenance, where they read vibration and temperature sequences from a compressor and raise an alert when the pattern diverges from normal. Older speech recognition and machine translation systems were recurrent too, and some still run in production.
What to watch for
- For new language projects, a transformer is almost always the better choice, while RNNs remain reasonable for compact time-series models on modest hardware.
- Sequence models are sensitive to how far back they can see, so test different input window lengths rather than guessing.
- Recurrent inference is sequential and cannot be parallelized across time steps, which limits throughput on long sequences.
- Compare against a simple statistical baseline such as seasonal averaging before accepting a neural forecast.