Max Wolf's Second Brain

      • activation space
      • actor critic
      • advantage
      • agency
      • agent
      • algebra
      • astrocyte
      • attention (general)
      • attention (ML)
      • Automating the Search for Artificial Life with Foundation Models
      • Bayes Theorem
      • beauty
      • bernoulli
      • bias
      • bias-variance tradeoff
      • bijective
      • binomial coefficient
      • binomial distribution
      • bootstrapping
      • borel set
      • cartesian product
      • causal attention
      • cellular automata
      • central moment
      • chain rule of probability
      • closure
      • cma-es
      • column space
      • combinatorics
      • computational irreducibility
      • conditional probability
      • congruence
      • congruence class
      • consciousness
      • Consciousness as a coherence-inducing operator - Cosciousness is virtual
      • continuous
      • correlation matrix
      • coset
      • covariance
      • credit assignment
      • critical state
      • cross product
      • cross-attention
      • cross-entropy
      • cross-entropy loss
      • cumulative distribution function
      • curiosity
      • Curiosity-driven Exploration by Self-supervised Prediction
      • cyclic group
      • definitions
      • determinant
      • diagonal matrix
      • differentiability
      • dot product
      • DQN
      • echo state network
      • eigendecomposition
      • eigenspace
      • eigenvalue
      • eligibility trace
      • Embracing curiosity eliminates the exploration-exploitation dilemma
      • Encoding innate ability through a genomic bottleneck
      • EngramNCA - A Neural Cellular Automaton Model of Memory Transfer
      • epsilon greedy
      • equivariance
      • Evolution Strategies as a Scalable Alternative to Reinforcement Learning
      • evolutionary optimization
      • expected value
      • extended euclidian algorithm
      • factor group
      • factorization
      • first order optimization
      • fisher information
      • frobenius norm
      • function
      • gaussian elimination
      • goal
      • graph
      • High-Dimensional Continuous Control Using Generalized Advantage Estimation
      • homomorphism
      • HyperNEAT
      • Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity
      • independent
      • indirect encoding
      • inductive bias
      • initialization
      • intelligence
      • intrinsic curiosity module
      • Introduction to RL
      • invariance
      • inverse
      • inverse matrix
      • isomorph
      • isotropic
      • isotropic gaussian
      • KL-divergence
      • Large Memory Layers with Product Keys
      • law of total probability
      • learning
      • LeCun Initialization
      • life
      • likelihood
      • line segment
      • linear least squares regression
      • linear systems of equations
      • lipschitz continuity
      • liquid state machine
      • log-likelihood
      • log-sum-exp trick
      • logarithm
      • loss
      • machine
      • markov chain monte carlo
      • Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
      • matrix
      • matrix minor
      • mean
      • mean absolute error
      • mean squared error
      • meaning of life
      • memory
      • message passing
      • mind
      • Mixture of A Million Experts
      • mode connectivity
      • momentum
      • monte carlo methods
      • monte carlo tree search
      • Motif - Intrinsic Motivation from Artificial Intelligence Feedback
      • Multi-Agent Advantage Decomposition Theorem
      • Multi-Agent Reinforcement Learning is a Sequence Modeling Problem
      • multi-head attention
      • multivariate gaussian distribution
      • negative log-likelihood loss
      • neighbourhood
      • Neural Ordinary Differential Equations - Paper
      • neuron
      • neutral element
      • norm
      • normal subgroup
      • note-taking
      • null space
      • obsidian markdown features
      • On the Measure of Intelligence
      • Open-Endedness is Essential for Artificial Superhuman Intelligence
      • order
      • orthogonal
      • orthogonal complement
      • Pascal's triangle
      • permutation
      • permutation equivariance
      • permutation invariance
      • pivot
      • poisson distribution
      • policy
      • policy gradient
      • policy gradient theorem
      • policy iteration
      • potentiation
      • PPO
      • preference-based RL
      • preimage
      • probability
      • probability density function
      • probability distribution
      • probability mass function
      • pseudo inverse
      • Q-Learning
      • Q-value
      • random variable
      • random walk
      • rank
      • REINFORCE
      • reinforcement learning
      • reliability
      • reproduction
      • Requirements for self organization
      • reservoir computing
      • Reservoir Computing - A New Paradigm for Neural Networks
      • ResNet
      • robust
      • roots of unity
      • row space
      • sampling
      • SARSA
      • scale-free
      • scaled dot product attention
      • score function
      • second moment
      • self-attention
      • sensitivity
      • set
      • simulated annealing
      • singular value decomposition
      • softmax
      • songs of life and mind
      • span
      • SPARTA - Distributed Training with Sparse Parameter Averaging
      • specificity
      • spectral radius
      • spline
      • standard normal distribution
      • state-value
      • stationary distribution
      • statistics
      • Streaming Deep Reinforcement Learning Finally Works
      • subgroup
      • surjective
      • surprise
      • symmetric
      • symmetric group
      • symmetric matrix
      • TD Lambda
      • temporal difference learning
      • test
      • the big bang
      • The Bitter Lesson
      • the second brain
      • The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games
      • The Unbearable Slowness of Being
      • transformer
      • Transformer Squared - Self-Adaptive LLMs
      • translation invariance
      • TRPO
      • Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning
      • TU-Wien ADM Übungen
      • unitary
      • value function
      • value iteration
      • variance
      • vector
      • weight space
    Home

    ❯

    general

    ❯

    The Bitter Lesson

    The Bitter Lesson

    Apr 27, 20251 min read

    Models that are carefully designed to capture known phenomena (inductive bias) in the data can be counterproductive. Richard Sutton advocates focusing solely on scale, learning and search.


    Graph View

    Backlinks

    • transformer

    Created with Quartz v4.4.0 © 2025

    • GitHub
    • Discord Community