A statistical model is autoregressive if it predicts future values based on past values. For example, an autoregressive model might seek to predict a stock’s future prices based on its past performance.
Deep autoregressive models are sequence models, yet feed-forward (i.e. not recurrent); generative models, yet supervised. They are a compelling alternative to RNNs for sequential data, and GANs for generation tasks.
Autoregressive Models in Deep Learning — A Brief Survey
prediction
LLM
The Pitfalls of Next-Token Prediction
Anthropic Interpretability - Understanding how AI models think goes a bit into pitfalls of NTP