Exploding gradient is a problem in training neural networks with gradient-based learning methods. In this issue, the gradient becomes too large and error gradients explode during backpropagation. This leads to unstable training and it's difficult to land on a good solution. This problem usually occurs in deep networks with long sequences and can be mitigated through gradient clipping or other normalization techniques.

This issue of exploding gradients often leads to the values in the matrices of the artificial neural networks to reach infinity or NaN, making the network unable to learn effectively. The model's weights become so large that they overshadow any other input, leading to the model only outputting a particular class or value, regardless of input.

The exploding gradient problem is more frequent in recurrent neural networks (RNNs), especially those dealing with longer sequences of data. This is because the gradients in such networks are accumulated through time, leading to an explosion if not properly managed.

There are several techniques to mitigate this problem. One common method is gradient clipping, which places a predefined limit on the size of the gradient. This prevents the gradient from becoming too large and causing numerical overflow. Another technique is normalization, such as batch normalization or layer normalization, which ensures that the network's activations do not become too high or too low at any point in the network.

Vanishing gradients, which is the opposite problem where gradients become too small, can also occur in deep neural networks. Both issues often need to be addressed when training deep learning models to ensure stable and efficient learning.