Inner Products and Duality

Today we’re talking about structures and ideas that go together with complex vector spaces.

To be concrete, let’s take $\mathbb{C}^2$ : the space of two-dimensional vectors with complex entries. If this is your first foray into complex vector spaces, you can relax. They work just like real ones. At least, they will today, with one, important exception.

An inner or dot product on a complex vector space leaves room for a little more nuance than its real counter part. And confronting that nuance is our main goal for today.

Two Inner Products

Typically, an inner product means taking a pair of vectors, multiplying their components - one by one - and adding them together. If $x$ and $y$ are vectors in $\mathbb{C}^2$, the inner product of $x$ with $y$ is going to be

\begin{equation}\label{ip1}(x,y) = x_{1} y_{1} + x_{2} y_{2}. \end{equation}

Well that’s one kind of inner product, anyway. It’s the dot product familiar from freshman physics and the real numbers. You can use that if you like, but it’s a little unnatural in a complex setting.

Here’s another option:

\begin{equation}\label{ip2} (x,y) = \bar{x}_{1} y_{1} + \bar{x}_{2} y_{2}. \end{equation}

$\bar{x}_{1}$ being the complex conjugate of the complex number, $x_{1}$. Ditto $x_{2}$.

So we have two possible inner products. Which means we have a question: Which is the better inner product? Which one should we choose?

Well… I could talk about the concept of “duality”: at this stage and mention the obvious involutive automorphism of $\mathbb{C}$ made manifest by the complex conjugate. But. patience. First things first.

A better question to ask is: WHY do we need an inner product in the first place?

Wait, Why Inner Products?

We often use an inner product - especially for simple vector spaces isomorphic to spaces like $\mathbb{C}^{N}$ or $\mathbb{R}^{N}$ - as a way to compare vectors. To project one vector along another. Another important use is compute their magnitude. And it’s the last of these that we’re most interested in.

Given some complex vector $z$. What is the inner product of $z$ with itself? Well, the first of our two inner products $(1)$ would just be:

\begin{equation} (z,z) = z_{1}^{2} + z_{2}^{2};\quad z_{1},z_{2}\in\mathbb{C}.\end{equation}

And that’s… still a complex number. Big surprise. I know. But it’s a worth call out.

You see, since $z_{1}$ and $z_{2}$ can both have imaginary parts, their sum of squares will too. Which is not really a magnitude. It’s more of a point. And this is where you might compare the complex number field $\mathbb{C}$ with $\mathbb{R}^2$.

So what about that second inner product, $(2)$? What is $z$ with itself there? Well. Here $z$ with $z$ is:

\begin{equation}\label{zz} (z,z) = \bar{z}_{1}z_{1} + \bar{z}_{2}z_{2} = |z_{1}|^2 + |z_{2}|^2,\end{equation}

which is a positive, real number. It’s a “magnitude” in a true sense. Thus, $(2)$ and $(4)$ give us a real sense of “distance” from the origin. And so that is our inner product of choice for complex vector space.

Here’s a quick exercise to test yourself!

Think of the complex numbers as $\mathbb{R}^{2}$, a two-dimensional, Euclidean vector space. Let $z$ be a complex number, that is a “vector” in $\mathbb{C}$. Show that the act of taking the modulus - $|z| = \bar{z}z$ - is the same thing as taking the traditional, euclidean “dot” product in $\mathbb{R}^{2}$. You know: $x^{2} + y^{2}$.

The Default Inner Product

So. We’ve got an inner product on a complex vector space that gives us the ability to compute magnitudes. To simplify our discussion going forward let’s agree on what the definition of an inner product is once and for all.

The default inner product on a complex vector space $V$, is map from a pair of vectors, $x,y\in V$ to $\mathbb{C}$:

\begin{equation}\langle x,y\rangle = \sum_{i=1}^{N} \bar{x}_{i}y_{i},\end{equation}

Here of course, $x_{i}\in\mathbb{C}$ are the individual components of the complex vector $x\in\mathbb{C}$.

Here’s another quick exercise to test your understanding

Prove that the default inner product of a vector with itself is always a positive, real number.

The Adjoint and the Composition

You may have noticed that the default inner product is… kind of reducible. There are two steps, the first is “take complex conjugates” and the second is “multiply components and add”. Viewed from this perspective, we can generalize the inner product dramatically, and also learn something deep about the mathematics involved.

To emphasis this structure, let’s name these steps. Step one or “take complex conjugates” will be called “the Adjoint”. Step two or “multiply components and add” will be called “the Composition”.

The Adjoint shows up all the time in mathematics. Two familiar examples include: multiplying by minus one, or taking the complex conjugate. Less familiar, perhaps is reflection by a mirror or the compliment of a subset. All of these examples belong to a class of operations known as involutions: you do it twice and you’re back to where you started.

The Adjoint is an involution, and as such, it gives rise to the important notion of duality. Consider some vector $x$. $x$’s adjoint is its “dual” vector. To avoid confusing it with complex conjugation, we typically denote the adjoint of a vector $z$ by $z^{\dagger}$.

Dual Spaces

Now, here’s where things get a little abstract, and therefore, a little tricky. Let $V$ be some vector space. The collection of all “dual” or adjoint vectors itself forms a vector space. This dual space is defined by $V$ and the adjoint operation. Sometimes V and its dual are identical.Tthat is, they’re isomorphic as vector spaces. But as we’ll see shortly, sometimes they’re not. And to really bring all this together, we need to ask another question:

“Dual space are dual with respect to WHAT?”

The answer? The Composition. You know, the thing that both $(1)$ and $(2)$ have in common. The sum of products of elements.

For a complex vector space, the default inner product required a composition that resulted in a single complex number. We also required the ability to compute a vector’s magnitude, which is a real, positive number. Thus, we formed the Adjoint by complex conjugating each component of the vector.

The Composition is defined by the the structure of its associated vector space. If $V$ is $\mathbb{C}^{N}$, then great! The dual space will also be $\mathbb{C}^{N}$.

But $V$ might be the space of all $(12 x 3)$ - dimensional rectangular matrices with complex entries. In that case, the dual space would be the space of all $(3 x 12)$ - dimensional rectangular matrices with complex entries, and the adjoint would be a complex-conjugate AND a transpose.

Indeed, we can think of $\mathbb{C}^{2}$ as a space of column vectors: $(2 \times 1)$- dimensional complex matrices:

$$z\in \mathbb{C}^2; \quad z = \left(\begin{array}{c} z_{1} \\ z_{2}\end{array}\right).$$

Then its adjoint will be the row vectors, $(1 \times 2)$ - dimensional complex matrices. The adjoint will transpose the column to rows and take the complex conjugate:

$$z = \left(\begin{array}{c} z_{1} \\ z_{2}\end{array}\right) \rightarrow z^{\dagger} = \left(\bar{z}_{1} , \bar{z}_{2}\right).$$

And the Composition gives the instructions for how to multiply those structures together.

As with everything in mathematics, there are a lot of layers of structure we can add - or structure we can leave out. But some structures are more useful - arguably more natural - than others.

TL;DR

Today, we talked about inner products on complex vector spaces, and specifically, how we can build an inner product that will always give us a meaningful magnitude or distance. First develop the Adjoint operation as a duality with respect to the Composition map.