15.3 Tensor Products of Maps

Section 02 built the space $V \otimes W$ . A construction on spaces is worth little until it acts on maps as well: the philosophy of the whole book has been objects first, then the maps between them. So given $S \colon V \to V'$ and $T \colon W \to W'$ , we want a single map $V \otimes W \to V' \otimes W'$ built from the two. This section builds it, computes its matrix, and discovers that the matrix is the Kronecker product of Chapter 8 — which pays both of the debts that Chapter 8 §04 left open. It ends with the identification $V^{*} \otimes W \cong \mathcal{L}(V, W)$ , which every later section of the chapter uses.

15.3.1. A map on each factor gives a map on the product

The definition writes itself, and that is the danger. We want a map that sends $\mathbf{v} \otimes \mathbf{w}$ to $S\mathbf{v} \otimes T\mathbf{w}$ , so the temptation is to write

“Define $(S \otimes T)(\mathbf{v} \otimes \mathbf{w}) \mathrel{:=} S\mathbf{v} \otimes T\mathbf{w}$ , and extend linearly.”

Nothing in that sentence is a definition. Simple tensors span $V \otimes W$ (Proposition 15.2.4), but they do not exhaust it, and worse, the expression of an element as a combination of them is wildly non-unique: $\mathbf{v} \otimes \mathbf{w} = (2\mathbf{v}) \otimes (\tfrac12\mathbf{w})$ , and $\mathbf{v} \otimes \mathbf{w} + \mathbf{v} \otimes \mathbf{w}' = \mathbf{v} \otimes (\mathbf{w} + \mathbf{w}')$ . A formula prescribed on simple tensors may therefore assign two different values to one element, and then it defines nothing at all. Here is a formula that really does fail. Try to define $g \colon V \otimes W \to V$ by $g(\mathbf{v} \otimes \mathbf{w}) \mathrel{:=} \mathbf{v}$ . Taking $\mathbf{w} = \mathbf{0}$ gives $\mathbf{v} \otimes \mathbf{0} = \mathbf{0}$ (Proposition 15.2.8), so $g$ would have to send $\mathbf{0}$ to every $\mathbf{v}$ at once.

To build a map out of $V \otimes W$ , never write a formula on simple tensors: write down a bilinear map and let the universal property produce the linear one.

That is the discipline of this chapter, and applied here it costs three lines.

Theorem 15.3.1 (Tensor Product of Two Linear Maps)

Let $V, V', W, W'$ be vector spaces over $F$ , and let $S \in \mathcal{L}(V, V')$ and $T \in \mathcal{L}(W, W')$ . Then there is exactly one linear map $f \colon V \otimes W \to V' \otimes W'$ with $f(\mathbf{v} \otimes \mathbf{w}) = S\mathbf{v} \otimes T\mathbf{w}$ for all $\mathbf{v} \in V$ and $\mathbf{w} \in W$ .

Idea. The assignment $(\mathbf{v}, \mathbf{w}) \mapsto S\mathbf{v} \otimes T\mathbf{w}$ is bilinear, because $S$ and $T$ are linear and $\otimes$ is bilinear in its two slots. A bilinear map out of $V \times W$ is exactly what the universal property consumes, and what it returns is a unique linear map out of $V \otimes W$ . There is nothing else to check.

Proof. Define $\beta \colon V \times W \to V' \otimes W'$ by $\beta(\mathbf{v}, \mathbf{w}) \mathrel{:=} S\mathbf{v} \otimes T\mathbf{w}$ . Fix $\mathbf{w} \in W$ . The map $\mathbf{v} \mapsto S\mathbf{v} \otimes T\mathbf{w}$ is the composite of $S$ , which is linear, with the map $\mathbf{u} \mapsto \mathbf{u} \otimes T\mathbf{w}$ , which is linear because the map $\otimes \colon V' \times W' \to V' \otimes W'$ of Definition 15.2.1 is bilinear, hence linear in its first slot with the second slot held fixed. A composite of linear maps is linear (Theorem 2.4.2), so $\beta$ is linear in its first argument. Fixing $\mathbf{v}$ instead and swapping the roles of the two slots gives linearity in the second. Hence $\beta$ is bilinear.

By the universal property in Definition 15.2.1 there is a unique linear $f \colon V \otimes W \to V' \otimes W'$ with $f(\mathbf{v} \otimes \mathbf{w}) = \beta(\mathbf{v}, \mathbf{w}) = S\mathbf{v} \otimes T\mathbf{w}$ for all $\mathbf{v}, \mathbf{w}$ . This proves both existence and uniqueness.

Uniqueness is what licenses a name.

Definition 15.3.1 (Tensor product of linear maps)

Let $S \in \mathcal{L}(V, V')$ and $T \in \mathcal{L}(W, W')$ . The tensor product of $S$ and $T$ is the unique linear map $S \otimes T \colon V \otimes W \to V' \otimes W'$ of Theorem 15.3.1, characterized by $(S \otimes T)(\mathbf{v} \otimes \mathbf{w}) = S\mathbf{v} \otimes T\mathbf{w} \quad (\mathbf{v} \in V,\ \mathbf{w} \in W).$

The symbol $\otimes$ now has three jobs — a space, a vector, a map — and they agree: $S \otimes T$ applied to $\mathbf{v} \otimes \mathbf{w}$ is $S\mathbf{v} \otimes T\mathbf{w}$ , which is the only thing the notation could mean.

Examples.

The identity. $\operatorname{id}_V \otimes \operatorname{id}_W$ sends $\mathbf{v} \otimes \mathbf{w}$ to $\mathbf{v} \otimes \mathbf{w}$ , and so does $\operatorname{id}_{V \otimes W}$ . Both are linear and both are the factorization of the same bilinear map, so $\operatorname{id}_V \otimes \operatorname{id}_W = \operatorname{id}_{V \otimes W}$ by the uniqueness clause. Degenerate, and used in almost every proof below.
Acting on one slot. $S \otimes \operatorname{id}_W$ sends $\mathbf{v} \otimes \mathbf{w}$ to $S\mathbf{v} \otimes \mathbf{w}$ : it does $S$ in the first slot and leaves the second alone. If $V = W = F[x]$ and $S = D$ is differentiation, $D \otimes \operatorname{id}$ is “differentiate the first factor”.
Scalars. For $c, d \in F$ , $(c\,\operatorname{id}_V) \otimes (d\,\operatorname{id}_W)$ sends $\mathbf{v} \otimes \mathbf{w}$ to $c\mathbf{v} \otimes d\mathbf{w} = cd\,(\mathbf{v} \otimes \mathbf{w})$ , so it is $cd\,\operatorname{id}_{V \otimes W}$ . In particular a scalar may be moved from one slot to the other, which is the map-level shadow of $c\mathbf{v} \otimes \mathbf{w} = \mathbf{v} \otimes c\mathbf{w}$ .
Zero. If $S = 0$ , then $(S \otimes T)(\mathbf{v} \otimes \mathbf{w}) = \mathbf{0} \otimes T\mathbf{w} = \mathbf{0}$ , so $S \otimes T = 0$ . The converse also holds; see Exercise (C1: When a tensor product of maps vanishes).

Non-example by minimal change. Keep $S$ and $T$ linear, but try to send $\mathbf{v} \otimes \mathbf{w}$ to $S\mathbf{v} \otimes T\mathbf{w} + \mathbf{v} \otimes \mathbf{w}$ . This one is legitimate: the assignment $(\mathbf{v}, \mathbf{w}) \mapsto S\mathbf{v} \otimes T\mathbf{w} + \mathbf{v} \otimes \mathbf{w}$ is a sum of two bilinear maps, hence bilinear, and the universal property applies. What is not legitimate is $(\mathbf{v}, \mathbf{w}) \mapsto S\mathbf{v} \otimes T\mathbf{w} + \mathbf{u}_0$ for a fixed non-zero $\mathbf{u}_0 \in V' \otimes W'$ : the added constant destroys linearity in each slot, so no linear map on $V \otimes W$ does this, and indeed the prescription contradicts itself at $\mathbf{w} = \mathbf{0}$ . The exact clause that fails is bilinearity, and it fails for the same reason as in the map $g$ above.

We will lean repeatedly on the following consequence of the uniqueness clause. It replaces the phrase “and extend linearly”, which we are not allowed to use.

Lemma 15.3.2 (Simple Tensors Determine a Map)

Let $Z$ be a vector space over $F$ and let $f, g \colon V \otimes W \to Z$ be linear. If $f(\mathbf{v} \otimes \mathbf{w}) = g(\mathbf{v} \otimes \mathbf{w})$ for all $\mathbf{v} \in V$ and $\mathbf{w} \in W$ , then $f = g$ .

Proof. Put $\beta(\mathbf{v}, \mathbf{w}) \mathrel{:=} f(\mathbf{v} \otimes \mathbf{w})$ . Since $\otimes$ is bilinear and $f$ is linear, $\beta$ is bilinear. By hypothesis $\beta(\mathbf{v}, \mathbf{w}) = g(\mathbf{v} \otimes \mathbf{w})$ as well, so $f$ and $g$ are both linear maps factoring $\beta$ through $\otimes$ . The universal property of Definition 15.2.1 says there is only one such map. Hence $f = g$ .

No finite-dimensionality is needed, and no basis is chosen. With it, the algebra of $\otimes$ on maps is a formality.

Proposition 15.3.3 (Algebra of Tensor Products of Maps)

Let $S, S_1, S_2 \in \mathcal{L}(V, V')$ , $T, T_1, T_2 \in \mathcal{L}(W, W')$ and $c \in F$ . Let also $R \in \mathcal{L}(V', V'')$ and $U \in \mathcal{L}(W', W'')$ . Then:

(a) $(S_1 + S_2) \otimes T = S_1 \otimes T + S_2 \otimes T$ and $S \otimes (T_1 + T_2) = S \otimes T_1 + S \otimes T_2$ ;
(b) $(cS) \otimes T = c(S \otimes T) = S \otimes (cT)$ ;
(c) $(R \otimes U)(S \otimes T) = RS \otimes UT$ ;
(d) $\operatorname{id}_V \otimes \operatorname{id}_W = \operatorname{id}_{V \otimes W}$ .

Proof. In each part both sides are linear maps out of $V \otimes W$ , so by Lemma 15.3.2 it suffices to compare their values at a simple tensor $\mathbf{v} \otimes \mathbf{w}$ .

(a) The left side gives $(S_1 + S_2)\mathbf{v} \otimes T\mathbf{w} = (S_1\mathbf{v} + S_2\mathbf{v}) \otimes T\mathbf{w}$ , which equals $S_1\mathbf{v} \otimes T\mathbf{w} + S_2\mathbf{v} \otimes T\mathbf{w}$ because $\otimes$ is additive in its first slot. That is the value of the right side. The second identity is the same argument in the other slot.
(b) $(cS)\mathbf{v} \otimes T\mathbf{w} = (c\,S\mathbf{v}) \otimes T\mathbf{w} = c(S\mathbf{v} \otimes T\mathbf{w}) = S\mathbf{v} \otimes (c\,T\mathbf{w})$ , where both outer equalities use bilinearity of $\otimes$ .
(c) $(R \otimes U)\big((S \otimes T)(\mathbf{v} \otimes \mathbf{w})\big) = (R \otimes U)(S\mathbf{v} \otimes T\mathbf{w}) = RS\mathbf{v} \otimes UT\mathbf{w}$ , which is the value of $RS \otimes UT$ .
(d) Done in Example 1.

Part (c) is the statement that $\otimes$ respects composition: it is functoriality, and we will see in a moment that it is the mixed product rule of Theorem 8.4.1 in disguise.

Two more structural facts are needed before §04 can speak of tensors with several slots. The first extends the universal property from two factors to any number, and the second says that the brackets do not matter.

For $k \ge 2$ define the iterated tensor product by $V_1 \otimes \dots \otimes V_k \mathrel{:=} (V_1 \otimes \dots \otimes V_{k-1}) \otimes V_k ,$ starting from $V_1$ itself when $k = 1$ , and write $\mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_k$ for $(\mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_{k-1}) \otimes \mathbf{v}_k$ . Multilinear maps of $k$ arguments were defined in Definition 15.1.1.

Lemma 15.3.4 (Universal Property for

k

Factors)

Let $V_1, \dots, V_k$ and $Z$ be vector spaces over $F$ , and let $\beta \colon V_1 \times \dots \times V_k \to Z$ be multilinear. Then there is exactly one linear map $\bar\beta \colon V_1 \otimes \dots \otimes V_k \to Z$ with $\bar\beta(\mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_k) = \beta(\mathbf{v}_1, \dots, \mathbf{v}_k)$ for all $\mathbf{v}_1, \dots, \mathbf{v}_k$ . Moreover the tensors $\mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_k$ span $V_1 \otimes \dots \otimes V_k$ .

Idea. Induction on $k$ , freezing the last argument. With $\mathbf{v}_k$ held fixed, $\beta$ is multilinear in the first $k-1$ arguments, so the inductive hypothesis converts it into a linear map on $X = V_1 \otimes \dots \otimes V_{k-1}$ . Letting $\mathbf{v}_k$ move again gives a bilinear map on $X \times V_k$ , and (T1) finishes. The spanning claim is the same induction.

Proof. For $k = 1$ take $\bar\beta = \beta$ , which is linear, and the claim is trivial; for $k = 2$ the statement is Definition 15.2.1 together with Proposition 15.2.4. Let $k \ge 3$ and assume the lemma for $k - 1$ . Put $X \mathrel{:=} V_1 \otimes \dots \otimes V_{k-1}$ .

Fix $\mathbf{v}_k \in V_k$ . The map $(\mathbf{v}_1, \dots, \mathbf{v}_{k-1}) \mapsto \beta(\mathbf{v}_1, \dots, \mathbf{v}_k)$ is multilinear, so the inductive hypothesis gives a unique linear $g_{\mathbf{v}_k} \colon X \to Z$ with $g_{\mathbf{v}_k}(\mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_{k-1}) = \beta(\mathbf{v}_1, \dots, \mathbf{v}_k)$ .

The map $X \times V_k \to Z$ , $(x, \mathbf{v}_k) \mapsto g_{\mathbf{v}_k}(x)$ , is bilinear. It is linear in $x$ because each $g_{\mathbf{v}_k}$ is. For linearity in $\mathbf{v}_k$ , fix $\mathbf{v}_k, \mathbf{v}'_k \in V_k$ and $c \in F$ : the linear maps $g_{\mathbf{v}_k + c\mathbf{v}'_k}$ and $g_{\mathbf{v}_k} + c\,g_{\mathbf{v}'_k}$ agree at every $\mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_{k-1}$ , because $\beta$ is linear in its last argument, and those tensors span $X$ by the inductive hypothesis; so the two maps are equal.

By Definition 15.2.1 there is a unique linear $\bar\beta$ on $X \otimes V_k = V_1 \otimes \dots \otimes V_k$ with $\bar\beta(x \otimes \mathbf{v}_k) = g_{\mathbf{v}_k}(x)$ ; taking $x = \mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_{k-1}$ gives $\bar\beta(\mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_k) = \beta(\mathbf{v}_1, \dots, \mathbf{v}_k)$ .

Finally, $X \otimes V_k$ is spanned by the simple tensors $x \otimes \mathbf{v}_k$ (Proposition 15.2.4), each $x$ is a combination of $\mathbf{v}_1 \otimes \dots \otimes \mathbf{v}_{k-1}$ by the inductive hypothesis, and $\otimes$ is linear in its first slot; so the $k$ -fold tensors span. A linear map is determined by its values on a spanning set, which gives uniqueness of $\bar\beta$ .

Proposition 15.3.5 (Associativity of the Tensor Product)

Let $U, V, W$ be vector spaces over $F$ . There is a unique isomorphism $\alpha \colon (U \otimes V) \otimes W \to U \otimes (V \otimes W)$ with $\alpha\big((\mathbf{u} \otimes \mathbf{v}) \otimes \mathbf{w}\big) = \mathbf{u} \otimes (\mathbf{v} \otimes \mathbf{w})$ for all $\mathbf{u}, \mathbf{v}, \mathbf{w}$ .

Proof. The map $(\mathbf{u}, \mathbf{v}, \mathbf{w}) \mapsto \mathbf{u} \otimes (\mathbf{v} \otimes \mathbf{w})$ is trilinear, since $\otimes$ is bilinear in each of its two uses. By Lemma 15.3.4 with $k = 3$ there is a unique linear $\alpha$ on $(U \otimes V) \otimes W$ with the stated values. Symmetrically, freezing the first argument instead and running the two-factor property twice produces a linear $\alpha' \colon U \otimes (V \otimes W) \to (U \otimes V) \otimes W$ with $\alpha'(\mathbf{u} \otimes (\mathbf{v} \otimes \mathbf{w})) = (\mathbf{u} \otimes \mathbf{v}) \otimes \mathbf{w}$ . The composite $\alpha'\alpha$ fixes every $(\mathbf{u} \otimes \mathbf{v}) \otimes \mathbf{w}$ , and those span by Lemma 15.3.4, so $\alpha'\alpha = \operatorname{id}$ ; symmetrically $\alpha\alpha' = \operatorname{id}$ . Hence $\alpha$ is an isomorphism.

From now on we drop the brackets and write $U \otimes V \otimes W$ , with $\mathbf{u} \otimes \mathbf{v} \otimes \mathbf{w}$ for the element that both ways of bracketing name, and likewise for any number of factors.

15.3.2. The matrix of a tensor product of maps

Now we compute. A matrix needs ordered bases, and $V \otimes W$ has a basis indexed by pairs $(i, k)$ , so the whole content of the computation is which order we put the pairs in.

Order the basis $\mathbf{v}_i \otimes \mathbf{w}_k$ by dictionary order on the pair $(i, k)$ : the index of the first factor changes slowly, the index of the second factor runs fastest.

Definition 15.3.2 (The product basis, ordered)

Let $\mathcal{B} = (\mathbf{v}_1, \dots, \mathbf{v}_n)$ be an ordered basis of $V$ and $\mathcal{C} = (\mathbf{w}_1, \dots, \mathbf{w}_q)$ an ordered basis of $W$ . Write $\mathcal{B} \otimes \mathcal{C}$ for the basis $(\mathbf{v}_i \otimes \mathbf{w}_k)$ of $V \otimes W$ given by Theorem 15.2.5, ordered so that $\mathbf{v}_i \otimes \mathbf{w}_k$ occupies position $(i - 1)q + k \qquad (1 \le i \le n,\ 1 \le k \le q),$ that is, in the order $\mathbf{v}_1 \otimes \mathbf{w}_1,\ \dots,\ \mathbf{v}_1 \otimes \mathbf{w}_q,\ \mathbf{v}_2 \otimes \mathbf{w}_1,\ \dots,\ \mathbf{v}_n \otimes \mathbf{w}_q .$

Division with remainder writes every position $1, \dots, nq$ as $(i-1)q + k$ for exactly one pair, so this really is an ordering of the whole basis. It is the dictionary order on pairs, and it is the same rule that Chapter 8 used to label the rows and columns of a Kronecker product in the formula $(\ast)$ after Definition 8.4.1. That is not a coincidence, and the next theorem says why.

Theorem 15.3.6 (The Kronecker Product Is the Matrix of a Tensor Product of Maps)

Let $V, V', W, W'$ be finite-dimensional vector spaces over $F$ with ordered bases $\begin{aligned} \mathcal{B} &= (\mathbf{v}_1, \dots, \mathbf{v}_n) \text{ of } V, &\quad \mathcal{B}' &= (\mathbf{v}'_1, \dots, \mathbf{v}'_m) \text{ of } V', \\ \mathcal{C} &= (\mathbf{w}_1, \dots, \mathbf{w}_q) \text{ of } W, &\quad \mathcal{C}' &= (\mathbf{w}'_1, \dots, \mathbf{w}'_p) \text{ of } W' . \end{aligned}$ Let $S \in \mathcal{L}(V, V')$ and $T \in \mathcal{L}(W, W')$ . Then, with the orderings of Definition 15.3.2 on both sides, $[S \otimes T]_{\mathcal{B} \otimes \mathcal{C}}^{\mathcal{B}' \otimes \mathcal{C}'} = [S]_{\mathcal{B}}^{\mathcal{B}'} \otimes [T]_{\mathcal{C}}^{\mathcal{C}'} ,$ the Kronecker product of Definition 8.4.1.

Idea. A matrix is read off column by column, and column $(j, l)$ of the left side holds the coordinates of $(S \otimes T)(\mathbf{v}_j \otimes \mathbf{w}_l) = S\mathbf{v}_j \otimes T\mathbf{w}_l$ . Expanding $S\mathbf{v}_j$ and $T\mathbf{w}_l$ in their bases and multiplying out by bilinearity produces the coefficient $a_{ij}b_{kl}$ on $\mathbf{v}'_i \otimes \mathbf{w}'_k$ — a product of one entry of each matrix. The entry formula $(\ast)$ of Chapter 8 says the Kronecker product puts exactly that number in exactly that place, provided the pairs are read in dictionary order. So the theorem is the entry formula and bilinearity, nothing more.

Proof. Write $\mathbf{A} = (a_{ij}) = [S]_{\mathcal{B}}^{\mathcal{B}'} \in M_{m \times n}(F)$ and $\mathbf{B} = (b_{kl}) = [T]_{\mathcal{C}}^{\mathcal{C}'} \in M_{p \times q}(F)$ , so that by Definition 4.1.1 $S\mathbf{v}_j = \sum_{i=1}^{m} a_{ij}\mathbf{v}'_i, \qquad T\mathbf{w}_l = \sum_{k=1}^{p} b_{kl}\mathbf{w}'_k .$ Fix $j$ and $l$ . Using Definition 15.3.1 and then bilinearity of $\otimes$ in each slot, $\begin{aligned} (S \otimes T)(\mathbf{v}_j \otimes \mathbf{w}_l) &= S\mathbf{v}_j \otimes T\mathbf{w}_l \\ &= \Big(\sum_{i} a_{ij}\mathbf{v}'_i\Big) \otimes \Big(\sum_{k} b_{kl}\mathbf{w}'_k\Big) \\ &= \sum_{i=1}^{m}\sum_{k=1}^{p} a_{ij}b_{kl}\,(\mathbf{v}'_i \otimes \mathbf{w}'_k) . \end{aligned}$ By Definition 15.3.2, $\mathbf{v}_j \otimes \mathbf{w}_l$ is the basis vector in position $(j-1)q + l$ of $\mathcal{B} \otimes \mathcal{C}$ , and $\mathbf{v}'_i \otimes \mathbf{w}'_k$ is the basis vector in position $(i-1)p + k$ of $\mathcal{B}' \otimes \mathcal{C}'$ . So the displayed expansion says that the entry of $[S \otimes T]_{\mathcal{B} \otimes \mathcal{C}}^{\mathcal{B}' \otimes \mathcal{C}'}$ in row $(i-1)p + k$ and column $(j-1)q + l$ is $a_{ij}b_{kl}$ .

By the entry formula $(\ast)$ accompanying Definition 8.4.1, that is exactly the entry of $\mathbf{A} \otimes \mathbf{B}$ in the same position. Every row index of an $mp \times nq$ matrix is $(i-1)p + k$ for exactly one pair $(i, k)$ , and every column index is $(j-1)q + l$ for exactly one pair $(j, l)$ , so the two matrices agree in every position. This proves the theorem.

The ordering is the whole content, so it is worth being exact about it. Suppose we had ordered the product basis the other way, putting $\mathbf{v}_i \otimes \mathbf{w}_k$ in position $(k-1)n + i$ , so that the second index changes slowly. Running the proof again, the entry in row $(k-1)m + i$ and column $(l-1)n + j$ is still $a_{ij}b_{kl}$ , which by $(\ast)$ is the entry of $\mathbf{B} \otimes \mathbf{A}$ in that position. So the other ordering gives $[T]_{\mathcal{C}}^{\mathcal{C}'} \otimes [S]_{\mathcal{B}}^{\mathcal{B}'} ,$ the Kronecker product with the factors swapped — not the transpose, and not the same matrix. For square factors the two answers are related exactly as Proposition 8.4.2 says, by conjugating with the permutation matrix that relabels dictionary order into its reverse — which is precisely the change-of-coordinates matrix between the two orderings of the product basis. In the rectangular setting of Theorem 15.3.6 the relation is not a conjugation at all: the row and column relabelings are different permutations, so $\mathbf{B} \otimes \mathbf{A} = \mathbf{P}_\sigma^{\top}(\mathbf{A} \otimes \mathbf{B})\mathbf{P}_\tau$ with two matrices, as Chapter 8 notes just after that proposition’s proof.

Warning. Do not reverse the two factors of the basis ordering. The theorem needs the index of the first tensor factor to be the slow one, matching Chapter 8’s dictionary order in $(\ast)$ . With the fast and slow indices exchanged the matrix of $S \otimes T$ is $[T] \otimes [S]$ . For $\mathbf{A} = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ and $\mathbf{B} = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$ these two differ in the $(1,2)$ entry, which is $a_{11}b_{12} = 1$ in the first and $b_{11}a_{12} = 0$ in the second (the non-example after Definition 8.4.1).

Chapter 8’s two promises, paid. At Definition 8.4.1, Chapter 8 wrote that the symbol $\otimes$ “anticipates the tensor product of Chapter 15, of which this is the matrix version”. Theorem 15.3.6 is that statement, made precise and proved: the Kronecker product of two matrices is the matrix of the tensor product of the two maps they represent, taken in the product bases. At the end of the same section, Chapter 8 wrote that the Kronecker product is “the coordinate shadow of a basis-free construction: the tensor product of linear maps, in Chapter 15, whose matrix in suitable bases is exactly $\mathbf{A} \otimes \mathbf{B}$ ”, and promised that “the dictionary order in $(\ast)$ will reappear there as an ordering of a basis $\mathbf{v}_i \otimes \mathbf{w}_k$ ”. The suitable bases are $\mathcal{B} \otimes \mathcal{C}$ and $\mathcal{B}' \otimes \mathcal{C}'$ ; the dictionary order has reappeared, as Definition 15.3.2; and the shadow is cast by $S \otimes T$ . The second promise gets a sharper payment at the end of this section, where the vec identity turns out to be this same theorem applied to a space of matrices.

Example (Computing with a tensor product of maps)

Let $S \colon F^2 \to F^2$ and $T \colon F^3 \to F^2$ have matrices, in the standard bases, $\mathbf{A} = \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}, \qquad \mathbf{B} = \begin{pmatrix} 1 & 0 & -1 \\ 2 & 1 & 0 \end{pmatrix}.$ Compute $(S \otimes T)(\mathbf{e}_1 \otimes \mathbf{f}_2 + \mathbf{e}_2 \otimes \mathbf{f}_3)$ directly from Definition 15.3.1, and again from the matrix of Theorem 15.3.6. Here $(\mathbf{e}_1, \mathbf{e}_2)$ is the standard basis of $F^2$ and $(\mathbf{f}_1, \mathbf{f}_2, \mathbf{f}_3)$ that of $F^3$ .

Solution. Directly. Columns of $\mathbf{A}$ and $\mathbf{B}$ give $S\mathbf{e}_1 = 2\mathbf{e}_1$ , $S\mathbf{e}_2 = \mathbf{e}_1 + 3\mathbf{e}_2$ , $T\mathbf{f}_2 = \mathbf{e}_2$ and $T\mathbf{f}_3 = -\mathbf{e}_1$ . Hence $\begin{aligned} (S \otimes T)(\mathbf{e}_1 \otimes \mathbf{f}_2 + \mathbf{e}_2 \otimes \mathbf{f}_3) &= 2\mathbf{e}_1 \otimes \mathbf{e}_2 + (\mathbf{e}_1 + 3\mathbf{e}_2) \otimes (-\mathbf{e}_1) \\ &= -\,\mathbf{e}_1 \otimes \mathbf{e}_1 + 2\,\mathbf{e}_1 \otimes \mathbf{e}_2 - 3\,\mathbf{e}_2 \otimes \mathbf{e}_1 , \end{aligned}$ using bilinearity of $\otimes$ in both slots.

By matrix. The domain basis, ordered as in Definition 15.3.2, is $(\mathbf{e}_1 \otimes \mathbf{f}_1, \mathbf{e}_1 \otimes \mathbf{f}_2, \mathbf{e}_1 \otimes \mathbf{f}_3, \mathbf{e}_2 \otimes \mathbf{f}_1, \mathbf{e}_2 \otimes \mathbf{f}_2, \mathbf{e}_2 \otimes \mathbf{f}_3)$ , so the input has coordinate column $(0, 1, 0, 0, 0, 1)$ . By Theorem 15.3.6 the matrix of $S \otimes T$ is $\mathbf{A} \otimes \mathbf{B} = \begin{pmatrix} 2 & 0 & -2 & 1 & 0 & -1 \\ 4 & 2 & 0 & 2 & 1 & 0 \\ 0 & 0 & 0 & 3 & 0 & -3 \\ 0 & 0 & 0 & 6 & 3 & 0 \end{pmatrix},$ and multiplying it by that column gives $(-1, 2, -3, 0)$ . In the codomain basis $(\mathbf{e}_1 \otimes \mathbf{e}_1, \mathbf{e}_1 \otimes \mathbf{e}_2, \mathbf{e}_2 \otimes \mathbf{e}_1, \mathbf{e}_2 \otimes \mathbf{e}_2)$ this is the same element as before.

15.3.3. What Chapter 8 was computing

Every property of the Kronecker product proved by entry-chasing in Chapter 8 is now a property of maps, and several of them become one-liners. We record the three that the rest of the chapter uses.

The mixed product rule comes first, because it is the one Chapter 8 called “the engine of everything that follows”.

Corollary 15.3.7 (The Mixed Product Rule Is Functoriality)

Let $\mathbf{A} \in M_{m \times n}(F)$ , $\mathbf{C} \in M_{n \times t}(F)$ , $\mathbf{B} \in M_{p \times q}(F)$ and $\mathbf{D} \in M_{q \times u}(F)$ . Then $(\mathbf{A} \otimes \mathbf{B})(\mathbf{C} \otimes \mathbf{D}) = (\mathbf{A} \mathbf{C}) \otimes (\mathbf{B} \mathbf{D}) .$

Proof. Let $S, R, T, U$ be the linear maps with matrices $\mathbf{A}, \mathbf{C}, \mathbf{B}, \mathbf{D}$ in the standard bases of the relevant spaces $F^k$ . By Theorem 15.3.6, with all product bases ordered as in Definition 15.3.2, $\mathbf{A} \otimes \mathbf{B}$ and $\mathbf{C} \otimes \mathbf{D}$ are the matrices of $S \otimes T$ and $R \otimes U$ . By Theorem 4.1.2 the product of the matrices is the matrix of the composite $(S \otimes T)(R \otimes U)$ , which is $SR \otimes TU$ by Proposition 15.3.3 (c). Applying Theorem 15.3.6 once more, and Theorem 4.1.2 again to each factor, the matrix of $SR \otimes TU$ is $(\mathbf{A} \mathbf{C}) \otimes (\mathbf{B} \mathbf{D})$ .

This is Theorem 8.4.1 (c), which Chapter 8 proved by partitioning both matrices into conformable blocks and multiplying blockwise. The proof above says what the identity means: doing two things one after the other in each slot separately is the same as doing them one after the other in both slots at once. In the same way, the bilinearity of Theorem 8.4.1 (a) is parts (a) and (b) of Proposition 15.3.3, its associativity (b) is Proposition 15.3.5, and its inverse rule (e) is parts (c) and (d) together with invertibility of a composite.

Next, the numerical invariants.

Corollary 15.3.8 (Rank, Determinant and Trace of a Tensor Product of Maps)

Let $V, V', W, W'$ be finite-dimensional over $F$ , $S \in \mathcal{L}(V, V')$ , $T \in \mathcal{L}(W, W')$ . Then $\operatorname{rank}(S \otimes T) = \operatorname{rank} S \cdot \operatorname{rank} T .$ If moreover $V = V'$ and $W = W'$ , with $n = \dim V \ge 1$ and $m = \dim W \ge 1$ , then $\det(S \otimes T) = (\det S)^{m}(\det T)^{n}, \qquad \operatorname{tr}(S \otimes T) = \operatorname{tr} S \cdot \operatorname{tr} T .$

Idea. For the rank, choose bases adapted to the two kernels: then $S \otimes T$ kills most product basis vectors and sends the remaining $\operatorname{rank} S \cdot \operatorname{rank} T$ of them to independent vectors. For the determinant, split $S \otimes T = (S \otimes \operatorname{id})(\operatorname{id} \otimes T)$ . The factor $\operatorname{id}_V \otimes T$ preserves each of the $n$ subspaces $\mathbf{v}_i \otimes W$ and acts on each as $T$ ; the factor $S \otimes \operatorname{id}_W$ preserves each of the $m$ subspaces $V \otimes \mathbf{w}_k$ and acts as $S$ . So each factor is block diagonal with repeated blocks, and the exponents count the other space’s dimension, which is where the exponents come from. The trace we take from the matrix.

Proof. Rank. Let $r = \operatorname{rank} S$ and $s = \operatorname{rank} T$ . Choose a basis $(\mathbf{v}_1, \dots, \mathbf{v}_n)$ of $V$ whose last $n - r$ vectors form a basis of $\operatorname{ker} S$ ; this is possible by Theorem 1.8.2 applied to a basis of $\operatorname{ker} S$ , after reordering. Then $S\mathbf{v}_1, \dots, S\mathbf{v}_r$ span $\operatorname{im} S$ by Theorem 2.2.4, and being $r = \operatorname{rank} S$ spanning vectors of an $r$ -dimensional space they form a basis of $\operatorname{im} S$ (Theorem 1.6.4). Choose a basis $(\mathbf{w}_1, \dots, \mathbf{w}_m)$ of $W$ the same way, so that $T\mathbf{w}_1, \dots, T\mathbf{w}_s$ is a basis of $\operatorname{im} T$ and $T\mathbf{w}_l = \mathbf{0}$ for $l > s$ .

By Theorem 2.2.4, $\operatorname{im}(S \otimes T)$ is spanned by the vectors $S\mathbf{v}_j \otimes T\mathbf{w}_l$ , and those with $j > r$ or $l > s$ are $\mathbf{0}$ . Extend $(S\mathbf{v}_1, \dots, S\mathbf{v}_r)$ to a basis of $V'$ and $(T\mathbf{w}_1, \dots, T\mathbf{w}_s)$ to a basis of $W'$ (Theorem 1.8.2). By Theorem 15.2.5 the products of these basis vectors form a basis of $V' \otimes W'$ ; in particular the $rs$ vectors $S\mathbf{v}_j \otimes T\mathbf{w}_l$ with $j \le r$ , $l \le s$ are linearly independent. They therefore form a basis of $\operatorname{im}(S \otimes T)$ , and $\operatorname{rank}(S \otimes T) = rs$ .

Determinant. By Proposition 15.3.3 (c) and (d), $S \otimes T = (S \otimes \operatorname{id}_W)(\operatorname{id}_V \otimes T),$ so $\det(S \otimes T) = \det(S \otimes \operatorname{id}_W)\det(\operatorname{id}_V \otimes T)$ by Theorem 7.5.8. Fix bases $\mathcal{B}$ of $V$ and $\mathcal{C}$ of $W$ and use the ordering of Definition 15.3.2. For each $i$ , the subspace $\mathbf{v}_i \otimes W \mathrel{:=} \operatorname{span}(\mathbf{v}_i \otimes \mathbf{w}_1, \dots, \mathbf{v}_i \otimes \mathbf{w}_m)$ is carried into itself by $\operatorname{id}_V \otimes T$ , which acts on it as $T$ does on $W$ ; and $V \otimes W$ is the direct sum of these $n$ subspaces, because their bases together form the basis $\mathcal{B} \otimes \mathcal{C}$ . Consecutive blocks of the ordering are exactly these subspaces, so $[\operatorname{id}_V \otimes T]_{\mathcal{B} \otimes \mathcal{C}}^{\mathcal{B} \otimes \mathcal{C}} = \underbrace{[T]_{\mathcal{C}}^{\mathcal{C}} \oplus \dots \oplus [T]_{\mathcal{C}}^{\mathcal{C}}}_{n \text{ blocks}},$ whose determinant is $(\det T)^{n}$ by Theorem 8.1.4 (d) and induction on the number of blocks. The same argument for $S \otimes \operatorname{id}_W$ , using instead the $m$ subspaces $V \otimes \mathbf{w}_k$ , gives $\det(S \otimes \operatorname{id}_W) = (\det S)^{m}$ . (This time the relevant basis vectors are not consecutive in the ordering of Definition 15.3.2, so reorder the basis to make them so; matrices of one operator in two bases are similar, by Theorem 4.2.3, hence have the same determinant, by Corollary 7.5.7.) Multiplying the two gives the formula.

Trace. By Theorem 15.3.6 the matrix of $S \otimes T$ in $\mathcal{B} \otimes \mathcal{C}$ is $[S]_{\mathcal{B}}^{\mathcal{B}} \otimes [T]_{\mathcal{C}}^{\mathcal{C}}$ , so by Definition 4.5.2 and Theorem 8.4.3 (b), $\operatorname{tr}(S \otimes T) = \operatorname{tr} S \cdot \operatorname{tr} T$ .

These are parts (a), (c) and (b) of Theorem 8.4.3, transported to operators. Two remarks on what has changed. First, the exponents in the determinant formula are no longer something to memorize: $\det S$ appears to the power $\dim W$ because $S \otimes \operatorname{id}_W$ runs $S$ once on each of the $\dim W$ copies of $V$ inside $V \otimes W$ . Second, the proof of the trace formula above is still a computation in a basis. There is a genuinely basis-free reason, and §04 gives it. The determinant, too, has a basis-free explanation, which has to wait for the top exterior power in §09.

Warning. Most operators on $V \otimes W$ are not of the form $S \otimes T$ . The operators that are of this form are heavily constrained: by Corollary 15.3.8 their rank is a product $rs$ with $r \le \dim V$ and $s \le \dim W$ . Take $V = W = F^2$ , so $\dim(V \otimes W) = 4$ , and let $P$ be any operator on $V \otimes W$ of rank $3$ , for instance the projection onto the span of the first three basis vectors of Definition 15.3.2. Then $3 = rs$ with $r, s \in \{0, 1, 2\}$ is impossible, so $P \ne S \otimes T$ for every $S$ and $T$ . The maps $S \otimes T$ do span $\mathcal{L}(V \otimes W)$ , which is a different and weaker statement; this is the same distinction as between simple tensors and general tensors in §02.

Quick check. Let $S \in \mathcal{L}(V)$ and $T \in \mathcal{L}(W)$ with $\dim V = 3$ and $\dim W = 2$ , and suppose $\det S = 2$ and $\det T = -1$ . What are $\det(S \otimes T)$ and $\det(T \otimes S)$ , where the second is an operator on $W \otimes V$ ?

Solution. By Corollary 15.3.8, $\det(S \otimes T) = (\det S)^{\dim W}(\det T)^{\dim V} = 2^{2}(-1)^{3} = -4$ . For $T \otimes S$ the roles swap, giving $(\det T)^{\dim V}(\det S)^{\dim W} = (-1)^{3}2^{2} = -4$ as well. The two agree, as they must: a determinant is a product of scalars, and the exponent always belongs to the other factor’s dimension. The tempting wrong answer, $(\det S)^{3}(\det T)^{2} = 8$ , puts each map’s own dimension in its exponent.

15.3.4. Maps as tensors

We come to the identification that the rest of the chapter runs on. A functional $\varphi \in V^{*}$ and a vector $\mathbf{w} \in W$ together determine a linear map $V \to W$ : measure with $\varphi$ , then scale $\mathbf{w}$ by the answer. That recipe is bilinear in $(\varphi, \mathbf{w})$ , which is now a reflex: it must come from a linear map on $V^{*} \otimes W$ .

Theorem 15.3.9 (Tensors Are Maps)

Let $V$ and $W$ be finite-dimensional vector spaces over $F$ . There is a unique linear map $\Theta \colon V^{*} \otimes W \to \mathcal{L}(V, W), \qquad \Theta(\varphi \otimes \mathbf{w})(\mathbf{v}) = \varphi(\mathbf{v})\,\mathbf{w} ,$ and it is an isomorphism. Moreover, for $R \in \mathcal{L}(V_0, V)$ and $U \in \mathcal{L}(W, W_0)$ with $V_0, W_0$ finite-dimensional, $\Theta\big((R' \otimes U)(\xi)\big) = U\,\Theta(\xi)\,R \qquad (\xi \in V^{*} \otimes W),$ where $R' \colon V^{*} \to V_0^{*}$ is the dual map of Definition 5.4.1 and the $\Theta$ on the left is the one for the pair $(V_0, W_0)$ .

Idea. Existence is the universal property again. For bijectivity, count: both sides have dimension $\dim V \cdot \dim W$ , so it is enough to see that $\Theta$ carries a basis onto a basis. It carries $\varphi_j \otimes \mathbf{w}_i$ to the map whose matrix is the matrix unit $\mathbf{E}_{ij}$ , and the matrix units are a basis. The last formula is the statement that $\Theta$ turns the tensor product of maps into composition on the two sides; it is checked on simple tensors.

Proof. Existence and uniqueness. For fixed $\varphi$ and $\mathbf{w}$ , the assignment $\mathbf{v} \mapsto \varphi(\mathbf{v})\mathbf{w}$ is linear, so it is an element $\varphi \cdot \mathbf{w}$ of $\mathcal{L}(V, W)$ . The map $(\varphi, \mathbf{w}) \mapsto \varphi \cdot \mathbf{w}$ is bilinear: for every $\mathbf{v}$ , $\begin{aligned} (\varphi + \psi)(\mathbf{v})\mathbf{w} &= \varphi(\mathbf{v})\mathbf{w} + \psi(\mathbf{v})\mathbf{w}, \\ \varphi(\mathbf{v})(\mathbf{w} + \mathbf{w}') &= \varphi(\mathbf{v})\mathbf{w} + \varphi(\mathbf{v})\mathbf{w}', \\ (c\varphi)(\mathbf{v})\mathbf{w} &= c\,\varphi(\mathbf{v})\mathbf{w} = \varphi(\mathbf{v})(c\mathbf{w}), \end{aligned}$ and two maps agreeing at every $\mathbf{v}$ are equal. By Definition 15.2.1 there is a unique linear $\Theta$ with $\Theta(\varphi \otimes \mathbf{w}) = \varphi \cdot \mathbf{w}$ .

Bijectivity. Let $\mathcal{B} = (\mathbf{v}_1, \dots, \mathbf{v}_n)$ be a basis of $V$ with dual basis $\mathcal{B}^{*} = (\varphi_1, \dots, \varphi_n)$ (Definition 5.1.3), and $\mathcal{C} = (\mathbf{w}_1, \dots, \mathbf{w}_m)$ a basis of $W$ . By Theorem 15.2.5 the products $\varphi_j \otimes \mathbf{w}_i$ form a basis of $V^{*} \otimes W$ , of size $nm$ . Now $\Theta(\varphi_j \otimes \mathbf{w}_i)$ sends $\mathbf{v}_k$ to $\varphi_j(\mathbf{v}_k)\mathbf{w}_i = \delta_{jk}\mathbf{w}_i$ , so it sends $\mathbf{v}_j$ to $\mathbf{w}_i$ and every other basis vector to $\mathbf{0}$ . By Definition 4.1.1 its matrix is the matrix unit $\mathbf{E}_{ij}$ . The matrix units form a basis of $M_{m \times n}(F)$ (Example (Standard Bases)) and $T \mapsto [T]_{\mathcal{B}}^{\mathcal{C}}$ is an isomorphism (Theorem 4.1.4), so the $nm$ maps $\Theta(\varphi_j \otimes \mathbf{w}_i)$ form a basis of $\mathcal{L}(V, W)$ .

Hence $\operatorname{im} \Theta$ contains a basis of $\mathcal{L}(V, W)$ , so $\Theta$ is surjective (Theorem 2.2.4). Since $\dim(V^{*} \otimes W) = nm = \dim \mathcal{L}(V, W)$ , Theorem 2.3.1 gives $\operatorname{nullity} \Theta = 0$ , so $\Theta$ is injective (Theorem 2.2.3). Therefore $\Theta$ is an isomorphism.

Naturality. Both $\xi \mapsto \Theta((R' \otimes U)\xi)$ and $\xi \mapsto U\,\Theta(\xi)\,R$ are linear in $\xi$ , so by Lemma 15.3.2 it suffices to check them at $\xi = \varphi \otimes \mathbf{w}$ . On the left, $(R' \otimes U)(\varphi \otimes \mathbf{w}) = R'\varphi \otimes U\mathbf{w}$ , and for $\mathbf{v} \in V_0$ , $\Theta(R'\varphi \otimes U\mathbf{w})(\mathbf{v}) = (R'\varphi)(\mathbf{v})\,U\mathbf{w} = \varphi(R\mathbf{v})\,U\mathbf{w} ,$ using Definition 5.4.1. On the right, $\big(U\,\Theta(\varphi \otimes \mathbf{w})\,R\big)(\mathbf{v}) = U\big(\varphi(R\mathbf{v})\mathbf{w}\big) = \varphi(R\mathbf{v})\,U\mathbf{w}$ by linearity of $U$ . The two agree.

Warning. Finite dimension is a hypothesis, not decoration. The map $\Theta$ is defined and injective for any $V$ and $W$ , but it is surjective only onto the maps of finite rank: $\Theta(\sum_{t=1}^{N} \varphi_t \otimes \mathbf{w}_t)$ has image inside $\operatorname{span}(\mathbf{w}_1, \dots, \mathbf{w}_N)$ , so its rank is at most $N$ . On $V = W = F[x]$ , the identity map has infinite rank and is therefore not $\Theta(\xi)$ for any $\xi \in V^{*} \otimes V$ . Everything below uses the finite-dimensional statement.

The letter $\Theta$ is deliberate: taking $V = F^n$ and $W = F^m$ and using the dual basis to identify $(F^n)^{*}$ with $F^n$ , this $\Theta$ is the isomorphism $F^m \otimes F^n \cong M_{m \times n}(F)$ of Theorem 15.2.7, with the two factors written in the order that makes the source a space of maps rather than of matrices. Section 02’s model was the special case; this is the general statement.

Two uses of the theorem, immediately. First, it names the matrix units: under $\Theta$ , the map with matrix $\mathbf{E}_{ij}$ is the tensor $\varphi_j \otimes \mathbf{w}_i$ , so a matrix is a sum $\sum_{i,j} a_{ij}\,\varphi_j \otimes \mathbf{w}_i$ . Note the index order — the functional carries the column index, the vector the row index — and compare it with Chapter 8’s vectorization, which stacks columns.

Second, the naturality formula is the vec identity.

Example (The vec identity, read as a tensor product of maps)

Let $\mathbf{A} \in M_{k \times m}(F)$ and $\mathbf{B} \in M_{n \times l}(F)$ , and let $\Psi \colon M_{m \times n}(F) \to M_{k \times l}(F), \qquad \mathbf{X} \mapsto \mathbf{A}\mathbf{X}\mathbf{B} ,$ which is linear. Identify $M_{m \times n}(F)$ with $\mathcal{L}(F^n, F^m)$ and $M_{k \times l}(F)$ with $\mathcal{L}(F^l, F^k)$ . Explain why Theorem 8.4.4, $\operatorname{vec}(\mathbf{A}\mathbf{X}\mathbf{B}) = (\mathbf{B}^{\top} \otimes \mathbf{A})\operatorname{vec}\mathbf{X}$ , is Theorem 15.3.6 applied to $\Psi$ .

Solution. Let $R \colon F^l \to F^n$ and $U \colon F^m \to F^k$ be the maps with matrices $\mathbf{B}$ and $\mathbf{A}$ in the standard bases. Under the identification, $\Psi$ becomes the map $\mathcal{L}(F^n, F^m) \to \mathcal{L}(F^l, F^k)$ sending $T$ to $UTR$ , so by the naturality clause of Theorem 15.3.9 it corresponds, through $\Theta$ , to $R' \otimes U \colon (F^n)^{*} \otimes F^m \to (F^l)^{*} \otimes F^k .$ Order the basis of $(F^n)^{*} \otimes F^m$ as in Definition 15.3.2, using the dual standard basis $(\varphi_1, \dots, \varphi_n)$ first and the standard basis $(\mathbf{e}_1, \dots, \mathbf{e}_m)$ second. Then $\varphi_j \otimes \mathbf{e}_i$ sits in position $(j-1)m + i$ , and $\Theta(\varphi_j \otimes \mathbf{e}_i) = \mathbf{E}_{ij}$ . By Definition 8.4.2, $\operatorname{vec} \mathbf{E}_{ij} = \mathbf{e}_{(j-1)m+i}$ . So $\operatorname{vec}$ is the coordinate map of this ordered basis: the dictionary order with the column index slow is exactly the order in which vectorization stacks the columns.

Now apply Theorem 15.3.6. The matrix of $R'$ in the dual standard bases is $\mathbf{B}^{\top}$ , because the matrix of a dual map is the transpose (Theorem 5.4.2), and the matrix of $U$ is $\mathbf{A}$ . Hence the matrix of $R' \otimes U$ in the product bases above is $\mathbf{B}^{\top} \otimes \mathbf{A}$ . Reading that equality of matrices through the coordinate map $\operatorname{vec}$ gives $\operatorname{vec}(\mathbf{A}\mathbf{X}\mathbf{B}) = (\mathbf{B}^{\top} \otimes \mathbf{A})\operatorname{vec}\mathbf{X}$ , which is Theorem 8.4.4.

For a numerical check, take $\mathbf{A} = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix}$ and $\mathbf{B} = \begin{pmatrix} 3 & 0 \\ 1 & -1 \end{pmatrix}$ . Writing the four images $\mathbf{A}\mathbf{E}_{ij}\mathbf{B}$ in the order $\mathbf{E}_{11}, \mathbf{E}_{21}, \mathbf{E}_{12}, \mathbf{E}_{22}$ and stacking the four columns $\operatorname{vec}(\mathbf{A}\mathbf{E}_{ij}\mathbf{B})$ side by side gives $\begin{pmatrix} 3 & 6 & 1 & 2 \\ 0 & 3 & 0 & 1 \\ 0 & 0 & -1 & -2 \\ 0 & 0 & 0 & -1 \end{pmatrix},$ which is $\mathbf{B}^{\top} \otimes \mathbf{A}$ written out.

So the transpose in the vec identity, which Chapter 8 had to warn readers not to get wrong, is not a bookkeeping accident: it is there because $\Psi$ multiplies $\mathbf{X}$ on the right by $\mathbf{B}$ , and right multiplication is precomposition, which reverses arrows. The dual map is what reverses them, and the matrix of a dual map is a transpose. That is the sharper form of Chapter 8’s second promise, and it is why the wrong guesses $\mathbf{A} \otimes \mathbf{B}^{\top}$ and $\mathbf{B} \otimes \mathbf{A}$ are wrong.

15.3.5. Exercises

A. Check your understanding

Exercise (A1)

(a) State the defining property of $S \otimes T$ , and say why writing “ $(S \otimes T)(\mathbf{v} \otimes \mathbf{w}) = S\mathbf{v} \otimes T\mathbf{w}$ , extended linearly” is not by itself a definition.
(b) Let $\dim V = 3$ , $\dim W = 4$ , $\dim V' = 2$ , $\dim W' = 5$ . What are the sizes of $[S]_{\mathcal{B}}^{\mathcal{B}'}$ , $[T]_{\mathcal{C}}^{\mathcal{C}'}$ and $[S \otimes T]_{\mathcal{B} \otimes \mathcal{C}}^{\mathcal{B}' \otimes \mathcal{C}'}$ ?
(c) In the ordering of Definition 15.3.2 with $\dim V = 3$ and $\dim W = 4$ , which basis vector of $V \otimes W$ sits in position $7$ ?
(d) True or false: every operator on $V \otimes W$ is of the form $S \otimes T$ . Justify your answer.
(e) State Theorem 15.3.9, including its hypothesis, and say what $\Theta(\varphi \otimes \mathbf{w})$ does to a vector $\mathbf{v}$ .

Solution.

(a) $S \otimes T$ is the unique linear map $V \otimes W \to V' \otimes W'$ with $(S \otimes T)(\mathbf{v} \otimes \mathbf{w}) = S\mathbf{v} \otimes T\mathbf{w}$ for all $\mathbf{v}, \mathbf{w}$ (Definition 15.3.1). The quoted phrase is not a definition because a general element of $V \otimes W$ is a sum of simple tensors in many different ways, so a prescription on simple tensors may be inconsistent; $\mathbf{v} \otimes \mathbf{0} = \mathbf{0}$ already forces consistency conditions. What makes it work is that $(\mathbf{v}, \mathbf{w}) \mapsto S\mathbf{v} \otimes T\mathbf{w}$ is bilinear, so Definition 15.2.1 supplies the map.
(b) $2 \times 3$ , $5 \times 4$ , and $10 \times 12$ (Theorem 15.3.6: an $m \times n$ and a $p \times q$ matrix give an $mp \times nq$ one).
(c) With $q = 4$ , position $7 = (i-1)\cdot 4 + k$ forces $i = 2$ , $k = 3$ . So it is $\mathbf{v}_2 \otimes \mathbf{w}_3$ .
(d) False. By Corollary 15.3.8, $\operatorname{rank}(S \otimes T) = \operatorname{rank} S \cdot \operatorname{rank} T$ . With $\dim V = \dim W = 2$ , an operator of rank $3$ on the $4$ -dimensional space $V \otimes W$ cannot have its rank factored as a product of two integers each at most $2$ .
(e) For $V, W$ finite-dimensional, there is a unique linear $\Theta \colon V^{*} \otimes W \to \mathcal{L}(V, W)$ with $\Theta(\varphi \otimes \mathbf{w})(\mathbf{v}) = \varphi(\mathbf{v})\mathbf{w}$ , and it is an isomorphism. So $\Theta(\varphi \otimes \mathbf{w})$ measures $\mathbf{v}$ with $\varphi$ and scales $\mathbf{w}$ by the result.

B. Practice

Exercise (B1: A tensor product of maps, two ways)

Let $S \colon \mathbb{Q}^2 \to \mathbb{Q}^2$ and $T \colon \mathbb{Q}^2 \to \mathbb{Q}^3$ have matrices, in the standard bases, $\mathbf{A} = \begin{pmatrix} 1 & -1 \\ 2 & 0 \end{pmatrix}, \qquad \mathbf{B} = \begin{pmatrix} 1 & 0 \\ 0 & 2 \\ -1 & 1 \end{pmatrix}.$

(a) Write down $[S \otimes T]_{\mathcal{B} \otimes \mathcal{C}}^{\mathcal{B}' \otimes \mathcal{C}'}$ for the standard bases, ordered as in Definition 15.3.2.
(b) Compute $(S \otimes T)(\mathbf{e}_1 \otimes \mathbf{e}_2 - \mathbf{e}_2 \otimes \mathbf{e}_1)$ directly from Definition 15.3.1.
(c) Check (b) against (a) by multiplying the matrix by the coordinate column of the input.

Solution.

(a) By Theorem 15.3.6 the matrix is $\mathbf{A} \otimes \mathbf{B}$ , of size $6 \times 4$ : $\mathbf{A} \otimes \mathbf{B} = \begin{pmatrix} 1 & 0 & -1 & 0 \\ 0 & 2 & 0 & -2 \\ -1 & 1 & 1 & -1 \\ 2 & 0 & 0 & 0 \\ 0 & 4 & 0 & 0 \\ -2 & 2 & 0 & 0 \end{pmatrix}.$
(b) The columns of $\mathbf{A}$ give $S\mathbf{e}_1 = \mathbf{e}_1 + 2\mathbf{e}_2$ and $S\mathbf{e}_2 = -\mathbf{e}_1$ ; the columns of $\mathbf{B}$ give $T\mathbf{e}_1 = \mathbf{f}_1 - \mathbf{f}_3$ and $T\mathbf{e}_2 = 2\mathbf{f}_2 + \mathbf{f}_3$ , where $(\mathbf{f}_1, \mathbf{f}_2, \mathbf{f}_3)$ is the standard basis of $\mathbb{Q}^3$ . Hence $\begin{aligned} &(S \otimes T)(\mathbf{e}_1 \otimes \mathbf{e}_2 - \mathbf{e}_2 \otimes \mathbf{e}_1) \\ &\quad = (\mathbf{e}_1 + 2\mathbf{e}_2) \otimes (2\mathbf{f}_2 + \mathbf{f}_3) - (-\mathbf{e}_1) \otimes (\mathbf{f}_1 - \mathbf{f}_3) \\ &\quad = \mathbf{e}_1 \otimes \mathbf{f}_1 + 2\,\mathbf{e}_1 \otimes \mathbf{f}_2 + 4\,\mathbf{e}_2 \otimes \mathbf{f}_2 + 2\,\mathbf{e}_2 \otimes \mathbf{f}_3 , \end{aligned}$ where the $\mathbf{e}_1 \otimes \mathbf{f}_3$ terms cancel: $+1$ from the first product and $-1$ from the second.
(c) The input has coordinates $(0, 1, -1, 0)$ in the basis $(\mathbf{e}_1 \otimes \mathbf{e}_1, \mathbf{e}_1 \otimes \mathbf{e}_2, \mathbf{e}_2 \otimes \mathbf{e}_1, \mathbf{e}_2 \otimes \mathbf{e}_2)$ . Multiplying gives $(1, 2, 0, 0, 4, 2)$ , which in the basis $(\mathbf{e}_1 \otimes \mathbf{f}_1, \mathbf{e}_1 \otimes \mathbf{f}_2, \mathbf{e}_1 \otimes \mathbf{f}_3, \mathbf{e}_2 \otimes \mathbf{f}_1, \mathbf{e}_2 \otimes \mathbf{f}_2, \mathbf{e}_2 \otimes \mathbf{f}_3)$ is exactly the answer to (b).

Exercise (B2: Invariants without multiplying matrices)

Let $S \in \mathcal{L}(V)$ and $T \in \mathcal{L}(W)$ over $\mathbb{R}$ , with $\dim V = 4$ , $\dim W = 3$ , $\operatorname{rank} S = 2$ , $\operatorname{rank} T = 3$ , $\operatorname{tr} S = 5$ , $\operatorname{tr} T = -2$ , $\det T = 6$ .

(a) Find $\operatorname{rank}(S \otimes T)$ and $\operatorname{nullity}(S \otimes T)$ .
(b) Find $\operatorname{tr}(S \otimes T)$ .
(c) Find $\det(S \otimes T)$ . Justify the value from the ranks alone.
(d) Find $\operatorname{tr}(S \otimes \operatorname{id}_W)$ and $\operatorname{tr}(\operatorname{id}_V \otimes T)$ , and check that their product is not $\operatorname{tr}(S \otimes T)$ in general by comparing with (b).

Solution.

(a) By Corollary 15.3.8, $\operatorname{rank}(S \otimes T) = 2 \cdot 3 = 6$ . Since $\dim(V \otimes W) = 12$ by Theorem 15.2.5, Theorem 2.3.1 gives $\operatorname{nullity}(S \otimes T) = 12 - 6 = 6$ .
(b) $\operatorname{tr}(S \otimes T) = 5 \cdot (-2) = -10$ .
(c) $\operatorname{rank} S = 2 < 4 = \dim V$ , so $S$ is not invertible and $\det S = 0$ by Theorem 3.4.4 and Definition 7.5.1. Hence $\det(S \otimes T) = 0^{3} \cdot 6^{4} = 0$ . Directly: $\operatorname{rank}(S \otimes T) = 6 < 12$ , so $S \otimes T$ is not invertible.
(d) $\operatorname{tr}(S \otimes \operatorname{id}_W) = \operatorname{tr} S \cdot \operatorname{tr} \operatorname{id}_W = 5 \cdot 3 = 15$ and $\operatorname{tr}(\operatorname{id}_V \otimes T) = 4 \cdot (-2) = -8$ , using $\operatorname{tr} \operatorname{id} = (\dim) \cdot 1$ from Theorem 4.5.3. Their product is $-120 \ne -10$ . There is no contradiction: $S \otimes T$ is the composite of the two, and the trace of a composite is not the product of the traces.

Exercise (B3: Which statements hold?)

Let $V, W$ be finite-dimensional over $F$ . Determine which of the following hold for all $S, S_1, S_2 \in \mathcal{L}(V)$ and $T \in \mathcal{L}(W)$ . Justify your answer.

(a) $(S_1S_2) \otimes T = (S_1 \otimes T)(S_2 \otimes T)$ .
(b) $(S_1S_2) \otimes T^2 = (S_1 \otimes T)(S_2 \otimes T)$ .
(c) $S \otimes T = 0$ implies $S = 0$ or $T = 0$ .
(d) $S \otimes \operatorname{id}_W$ and $\operatorname{id}_V \otimes T$ commute.
(e) If $S$ and $T$ are invertible, so is $S \otimes T$ , with inverse $S^{-1} \otimes T^{-1}$ .

Solution.

(a) Fails. By Proposition 15.3.3 (c) the right side is $S_1S_2 \otimes T^2$ , and with $S_1 = S_2 = \operatorname{id}_V$ and $T = 2\,\operatorname{id}_W$ over $\mathbb{Q}$ the two sides are $2\,\operatorname{id}$ and $4\,\operatorname{id}$ on $V \otimes W$ , by Example 3 after Definition 15.3.1.
(b) Holds: it is exactly Proposition 15.3.3 (c).
(c) Holds. If $S \ne 0$ and $T \ne 0$ , then $\operatorname{rank} S \ge 1$ and $\operatorname{rank} T \ge 1$ , so $\operatorname{rank}(S \otimes T) \ge 1$ by Corollary 15.3.8, and $S \otimes T \ne 0$ .
(d) Holds. By Proposition 15.3.3 (c) and (d), both composites equal $S \otimes T$ .
(e) Holds. By Proposition 15.3.3 (c) and (d), $(S \otimes T)(S^{-1} \otimes T^{-1}) = \operatorname{id}_V \otimes \operatorname{id}_W = \operatorname{id}_{V \otimes W}$ , and likewise in the other order.

C. Going deeper

Exercise (C1: When a tensor product of maps vanishes)

Let $V, V', W, W'$ be finite-dimensional over $F$ , $S \in \mathcal{L}(V, V')$ and $T \in \mathcal{L}(W, W')$ .

(a) Prove that $S \otimes T = 0$ if and only if $S = 0$ or $T = 0$ .
(b) Prove that $\operatorname{ker}(S \otimes \operatorname{id}_W) = \operatorname{ker} S \otimes W$ , meaning the span of all $\mathbf{v} \otimes \mathbf{w}$ with $\mathbf{v} \in \operatorname{ker} S$ .
(c) Deduce $\operatorname{nullity}(S \otimes \operatorname{id}_W) = \operatorname{nullity} S \cdot \dim W$ , and check it against Corollary 15.3.8.

Solution.

(a) $(\Leftarrow)$ If $S = 0$ , then $(S \otimes T)(\mathbf{v} \otimes \mathbf{w}) = \mathbf{0} \otimes T\mathbf{w} = \mathbf{0}$ for all $\mathbf{v}, \mathbf{w}$ , so $S \otimes T = 0$ by Lemma 15.3.2. Likewise if $T = 0$ . $(\Rightarrow)$ Suppose $S \ne 0$ and $T \ne 0$ . Then $\operatorname{rank} S \ge 1$ and $\operatorname{rank} T \ge 1$ , so $\operatorname{rank}(S \otimes T) = \operatorname{rank} S \cdot \operatorname{rank} T \ge 1$ by Corollary 15.3.8, and $S \otimes T \ne 0$ .
(b) $(\supseteq)$ For $\mathbf{v} \in \operatorname{ker} S$ and $\mathbf{w} \in W$ , $(S \otimes \operatorname{id}_W)(\mathbf{v} \otimes \mathbf{w}) = \mathbf{0} \otimes \mathbf{w} = \mathbf{0}$ ; a kernel is a subspace (Theorem 2.2.1), so it contains the span of these elements. $(\subseteq)$ Choose a basis $(\mathbf{v}_1, \dots, \mathbf{v}_n)$ of $V$ whose last $n - r$ vectors span $\operatorname{ker} S$ , where $r = \operatorname{rank} S$ , and such that $S\mathbf{v}_1, \dots, S\mathbf{v}_r$ are independent (as in the proof of Corollary 15.3.8). Let $(\mathbf{w}_1, \dots, \mathbf{w}_q)$ be a basis of $W$ and let $z = \sum_{j,l} c_{jl}\,\mathbf{v}_j \otimes \mathbf{w}_l$ lie in the kernel. Then $\mathbf{0} = (S \otimes \operatorname{id}_W)(z) = \sum_{j \le r}\sum_{l} c_{jl}\,S\mathbf{v}_j \otimes \mathbf{w}_l ,$ and the vectors $S\mathbf{v}_j \otimes \mathbf{w}_l$ with $j \le r$ are linearly independent by Theorem 15.2.5, after extending $(S\mathbf{v}_1, \dots, S\mathbf{v}_r)$ to a basis of $V'$ . Hence $c_{jl} = 0$ for all $j \le r$ , so $z$ is a combination of the $\mathbf{v}_j \otimes \mathbf{w}_l$ with $j > r$ , all of which lie in $\operatorname{ker} S \otimes W$ .
(c) By (b) and Theorem 15.2.5 applied to $\operatorname{ker} S$ and $W$ , $\operatorname{nullity}(S \otimes \operatorname{id}_W) = \dim \operatorname{ker} S \cdot \dim W = \operatorname{nullity} S \cdot \dim W$ . Consistency: Corollary 15.3.8 gives $\operatorname{rank}(S \otimes \operatorname{id}_W) = \operatorname{rank} S \cdot \dim W$ , and adding the two gives $(\operatorname{rank} S + \operatorname{nullity} S)\dim W = \dim V \cdot \dim W = \dim(V \otimes W)$ , as Theorem 2.3.1 requires.

Exercise (C2: Rank-one maps and simple tensors)

Let $V, W$ be finite-dimensional over $F$ and let $\Theta$ be the isomorphism of Theorem 15.3.9.

(a) Prove that $\Theta(\xi)$ has rank at most $1$ if and only if $\xi$ is a simple tensor $\varphi \otimes \mathbf{w}$ .
(b) Deduce that “not every tensor is simple” (§02) says exactly that not every linear map has rank at most $1$ , and identify the tensor corresponding to $\operatorname{id}_V$ in a basis.
(c) Let $\dim V = \dim W = 2$ . Exhibit $\xi \in V^{*} \otimes W$ that is not simple, by exhibiting a map of rank $2$ .

Solution.

(a) $(\Leftarrow)$ $\Theta(\varphi \otimes \mathbf{w})$ sends every $\mathbf{v}$ into $\operatorname{span}(\mathbf{w})$ , so its image has dimension at most $1$ . $(\Rightarrow)$ Suppose $R \mathrel{:=} \Theta(\xi)$ has rank at most $1$ . If $R = 0$ , then $\xi = 0 = 0 \otimes \mathbf{0}$ since $\Theta$ is injective. Otherwise $\operatorname{im} R = \operatorname{span}(\mathbf{w})$ for some $\mathbf{w} \ne \mathbf{0}$ , so for each $\mathbf{v}$ there is a unique scalar $\varphi(\mathbf{v})$ with $R\mathbf{v} = \varphi(\mathbf{v})\mathbf{w}$ ; uniqueness and linearity of $R$ make $\varphi$ linear, so $\varphi \in V^{*}$ . Then $R = \Theta(\varphi \otimes \mathbf{w})$ , and $\xi = \varphi \otimes \mathbf{w}$ because $\Theta$ is injective.
(b) By (a) and bijectivity of $\Theta$ , the non-simple tensors in $V^{*} \otimes W$ correspond exactly to the maps of rank at least $2$ . Taking $W = V$ with basis $\mathcal{B} = (\mathbf{v}_1, \dots, \mathbf{v}_n)$ and dual basis $(\varphi_1, \dots, \varphi_n)$ , the matrix of $\operatorname{id}_V$ is $\mathbf{I}_n = \sum_i \mathbf{E}_{ii}$ , so $\operatorname{id}_V = \Theta\big(\sum_{i=1}^{n} \varphi_i \otimes \mathbf{v}_i\big)$ . For $n \ge 2$ this has rank $n \ge 2$ , hence is not simple. For $n = 2$ it is the witness $\varphi_1 \otimes \mathbf{v}_1 + \varphi_2 \otimes \mathbf{v}_2$ , which is Example (A tensor that is not simple) read through $\Theta$ .
(c) Take $\xi = \varphi_1 \otimes \mathbf{w}_1 + \varphi_2 \otimes \mathbf{w}_2$ for bases $(\mathbf{v}_1, \mathbf{v}_2)$ of $V$ and $(\mathbf{w}_1, \mathbf{w}_2)$ of $W$ . Then $\Theta(\xi)$ sends $\mathbf{v}_1 \mapsto \mathbf{w}_1$ and $\mathbf{v}_2 \mapsto \mathbf{w}_2$ , so it has rank $2$ , and by (a) $\xi$ is not simple.

Exercise (C3: Reproving the swap)

Let $V$ and $W$ be finite-dimensional over $F$ with ordered bases $\mathcal{B}$ and $\mathcal{C}$ .

(a) Prove that there is a unique isomorphism $\tau \colon V \otimes W \to W \otimes V$ with $\tau(\mathbf{v} \otimes \mathbf{w}) = \mathbf{w} \otimes \mathbf{v}$ .
(b) Prove that $\tau(S \otimes T) = (T \otimes S)\tau$ for all $S \in \mathcal{L}(V)$ , $T \in \mathcal{L}(W)$ .
(c) Deduce Proposition 8.4.2: for $\mathbf{A} \in M_n(F)$ and $\mathbf{B} \in M_m(F)$ there is a permutation matrix $\mathbf{P}$ with $\mathbf{B} \otimes \mathbf{A} = \mathbf{P}^{-1}(\mathbf{A} \otimes \mathbf{B})\mathbf{P}$ , and identify $\mathbf{P}$ .

Hint for (c): think of $\tau$ as renaming basis vectors rather than as moving them.

Solution.

(a) The map $(\mathbf{v}, \mathbf{w}) \mapsto \mathbf{w} \otimes \mathbf{v}$ is bilinear, so Definition 15.2.1 gives a unique linear $\tau$ with $\tau(\mathbf{v} \otimes \mathbf{w}) = \mathbf{w} \otimes \mathbf{v}$ . The same construction with the roles swapped gives $\sigma \colon W \otimes V \to V \otimes W$ with $\sigma(\mathbf{w} \otimes \mathbf{v}) = \mathbf{v} \otimes \mathbf{w}$ . Then $\sigma\tau$ fixes every simple tensor, so $\sigma\tau = \operatorname{id}$ by Lemma 15.3.2, and likewise $\tau\sigma = \operatorname{id}$ . Hence $\tau$ is an isomorphism.
(b) Both sides are linear maps out of $V \otimes W$ , so by Lemma 15.3.2 compare at $\mathbf{v} \otimes \mathbf{w}$ : the left gives $\tau(S\mathbf{v} \otimes T\mathbf{w}) = T\mathbf{w} \otimes S\mathbf{v}$ , and the right gives $(T \otimes S)(\mathbf{w} \otimes \mathbf{v}) = T\mathbf{w} \otimes S\mathbf{v}$ .
(c) Let $S, T$ be the operators on $F^n, F^m$ with matrices $\mathbf{A}, \mathbf{B}$ . Both $\mathcal{B} \otimes \mathcal{C}$ and $\tau^{-1}(\mathcal{C} \otimes \mathcal{B})$ are orderings of the same basis of $F^n \otimes F^m$ , differing by the permutation $\sigma$ that sends position $(k-1)n + i$ to position $(i-1)m + k$ — the direction matters, since $\sigma$ is not an involution unless $n = m$ — and let $\mathbf{P}$ be its matrix, which is exactly the $\mathbf{P}_\sigma$ built in the proof of Proposition 8.4.2.