The relation the cosine angle between vectors and their inner product

This shows the plane triangle with vectors x, y, and y-x where the coordinate axis are defined in terms of the plane of vector x, or in three dimensions, the vector x on the relative plane of such coordinate axis can be thought as having x₁ = relative x component, x₂ = relative y component. Noticing that the direction y₃ a relative z component is perpendicular to component plane of x₁ and x₂.

First, I'll address why the inner product must be zero for two perpendicular vectors.

For two given vectors x = (x₁,x₂, …, x_n) and y = (y₁, y₂, …, y_n), it should be noted that two vectors are perpendicular if

This results from the Pythagorean formula where the magnitude (distances) of each side length of the vectors on the left side of the equation squared and summed equal the magnitude (distance) of of the vector x-y on the right side of the equation.  One would note the relation of vectors above with application to the Pythagorean formula, while the figure above demonstrates relations here in three dimensions, this application extends to n dimensions likewise.

Expanding the right side of the equation (1) we have

we'd note that expansion of the left side of equation (1) leads to cancellations of the terms

  

Thus we are left with the requirement for the given equality that the central group of terms in equation (2) are equal to zero, or the “dot product” (or inner product) of vectors x and y are orthogonal if

Now getting to the relation of the cosine angle between two vectors x and y.

We'll demonstrate the relation of the inner product using the figure above which is shown in two dimensions. Note this could be extended to n dimensions but for now we'll focus on two dimensions in deriving the relation between the cosine angle of two given vectors and an inner product.

From the figure above one can note the following relations,

  

Now using the relation θ = β - α and using the known trigonometric sum/difference identity and substituting the relations above we have

This can be generalized to any vectors of n dimensions such that

Sources compiled from Linear Algebra and Applications, Strang, 1988.