Machine Learning Encyclopedia

Correlation coefficient

The most well-known measure of similarity between two random variables is the correlation coefficient. Correlation coefficient $\rho$ between two random variables $X_1$ and $X_2$ is defined as

\begin{equation} \rho(X_1,X_2) = \frac{\text{cov}(X_1,X_2)}{\sqrt{\text{var}(X_1), \text{var}(X_2)}}.\nonumber \end{equation}

If $X_1$ and $X_2$ are completely correlated, i.e., exact linear dependency exists $\rho(X_1,X_2)$ is $1$ or $-1$. If $X_1$ and $X_2$ are totally uncorrelated, $\rho(X_1,X_2)$ is $0$.