isotropic distribution

A sphere is isotropic.
The CMB has (tiny) aniostropies.

Link to original

For matrices, this means being proportional to the identity matrix: $A=cI$ for some $c>0$

Isotropic Distribution

A multivariate distribution is isotropic if it is centered and its covariance matrix is proportional to the identity matrix:

$$\text{Cov}(X)=cI$$

for some $c>0$.
The variance in every direction is the same.
A special case is when the distribution is in isotropic position ($c=1$).

This does not mean the distribution is actually a sphere / rotationally symmetric

Spherical symmetry is a stronger condition than isotropy for distribution, since it requires the distribution to be invariant under any rotation, not just having equal variance in all directions.
However, for normal distributions, these two coincide: See isotropic gaussian.
See also: spherical distribution.
In physics, isotropic usually does mean spherical symmetry.

Diagonal matrices have nice computational properties and limit the growth to be linear instead of quadratic for non-diagonal matrices.

The proccess of turning a non-isotropic distribution into an isotropic one is called whitening.