Abstract¶ The conditions for a functional
Keywords: Norms Vector Spaces Vectors Matrices ¶ Norm Conditions ¶ Let V V V F \mathbb{F} F ∣ ∣ ⋅ ∣ ∣ \vert \vert \cdot \vert \vert ∣∣ ⋅ ∣∣ V V V ∣ ∣ ⋅ ∣ ∣ : V → R + \vert \vert \cdot \vert \vert: V \rightarrow \mathbb{R}^{+} ∣∣ ⋅ ∣∣ : V → R + ψ , σ ∈ V \psi, \sigma ~\in ~V ψ , σ ∈ V λ ∈ F 1 \lambda~\in~\mathbb{F}^{1} λ ∈ F 1
∣ ∣ ψ ∣ ∣ ≥ 0 \vert \vert \psi \vert \vert \geq 0 ∣∣ ψ ∣∣ ≥ 0 ∣ ∣ ψ ∣ ∣ = 0 \vert \vert \psi \vert \vert=0 ∣∣ ψ ∣∣ = 0 ψ = 0 \psi = 0 ψ = 0 ∣ ∣ λ ψ ∣ ∣ = ∣ λ ∣ ∣ ∣ ψ ∣ ∣ \vert \vert \lambda \psi \vert \vert = \vert \lambda \vert ~ \vert \vert \psi \vert \vert ∣∣ λ ψ ∣∣ = ∣ λ ∣ ∣∣ ψ ∣∣ ∣ ∣ ψ + σ ∣ ∣ ≤ ∣ ∣ ψ ∣ ∣ + ∣ ∣ σ ∣ ∣ \vert \vert \psi + \sigma \vert \vert \leq \vert \vert \psi \vert \vert + \vert \vert \sigma \vert \vert ∣∣ ψ + σ ∣∣ ≤ ∣∣ ψ ∣∣ + ∣∣ σ ∣∣ Triangle Inequality From this is can be seen that: ∣ ∣ ∣ ψ ∣ ∣ − ∣ ∣ σ ∣ ∣ ∣ ≤ ∣ ∣ ψ − σ ∣ ∣ \vert ~ \vert \vert \psi \vert \vert - \vert \vert \sigma \vert \vert ~\vert \leq \vert \vert \psi - \sigma \vert \vert ∣ ∣∣ ψ ∣∣ − ∣∣ σ ∣∣ ∣ ≤ ∣∣ ψ − σ ∣∣ Norms can be thought of as functionals that measure the size of an elements of a vector space. A vector space with a norm is called a normed vector space.
Vector Norm Examples ¶
Let ψ ∈ V \psi~\in~V ψ ∈ V ψ ∈ F n \psi~\in~\mathbb{F}^{n} ψ ∈ F n ( ψ 1 , ψ 2 , … , ψ n ) (\psi_1, \psi_2, \ldots, \psi_n) ( ψ 1 , ψ 2 , … , ψ n ) l p l_{p} l p
∣ ∣ ψ ∣ ∣ p = ( ∣ ψ 1 ∣ p + ∣ ψ 2 ∣ p + … + ∣ ψ n ∣ p ) 1 / p \vert \vert \psi \vert \vert_{p} = \big( \vert \psi_1 \vert^p + \vert \psi_2 \vert^p + \ldots + \vert \psi_n \vert^p)^{1/p} ∣∣ ψ ∣ ∣ p = ( ∣ ψ 1 ∣ p + ∣ ψ 2 ∣ p + … + ∣ ψ n ∣ p ) 1/ p With p = 1 p=1 p = 1 p = 2 p=2 p = 2 l 2 l_2 l 2
∣ ∣ ψ ∣ ∣ 1 = ( ∣ ψ 1 ∣ + ∣ ψ 2 ∣ + … + ∣ ψ n ∣ ) ∣ ∣ ψ ∣ ∣ 2 = ( ∣ ψ 1 ∣ 2 + ∣ ψ 2 ∣ 2 + … + ∣ ψ n ∣ 2 ) 1 / 2 \begin{align*}
\vert \vert \psi \vert \vert_{1} &= \big( \vert \psi_1 \vert + \vert \psi_2 \vert + \ldots + \vert \psi_n \vert) \\
\vert \vert \psi \vert \vert_{2} &= \big( \vert \psi_1 \vert^2 + \vert \psi_2 \vert^2 + \ldots + \vert \psi_n \vert^2)^{1/2}
\end{align*} ∣∣ ψ ∣ ∣ 1 ∣∣ ψ ∣ ∣ 2 = ( ∣ ψ 1 ∣ + ∣ ψ 2 ∣ + … + ∣ ψ n ∣ ) = ( ∣ ψ 1 ∣ 2 + ∣ ψ 2 ∣ 2 + … + ∣ ψ n ∣ 2 ) 1/2 Let ψ ∈ V \psi~\in~V ψ ∈ V ψ ∈ F n \psi~\in~\mathbb{F}^{n} ψ ∈ F n ( ψ 1 , ψ 2 , … , ψ n ) (\psi_1, \psi_2, \ldots, \psi_n) ( ψ 1 , ψ 2 , … , ψ n )
∣ ∣ ψ ∣ ∣ ∞ = max { ∣ ψ i ∣ : 1 ≤ i ≤ n } . \vert \vert \psi \vert \vert_{\infty} = \max \{\vert \psi_i \vert~: 1 \leq i \leq n \}. ∣∣ ψ ∣ ∣ ∞ = max { ∣ ψ i ∣ : 1 ≤ i ≤ n } . This is just the maximum elements of ψ \psi ψ
Matrix Norms ¶ In addition to satisfying the above requirements, to be considered a norm on a vector space, M M M Ω : M → R + \Omega: M \rightarrow \mathbb{R}^{+} Ω : M → R + A , B ∈ M A,B~\in~M A , B ∈ M
∣ ∣ A B ∣ ∣ ≤ ∣ ∣ A ∣ ∣ ∣ ∣ B ∣ ∣ \vert \vert AB \vert \vert \leq \vert \vert A \vert \vert ~ \vert \vert B \vert \vert ∣∣ A B ∣∣ ≤ ∣∣ A ∣∣ ∣∣ B ∣∣ Matrix Norm Examples ¶ The following norms depend only on the singular values of the matrices they act upon. This means that the following conditions are true:
∣ ∣ A ∣ ∣ = ∣ ∣ A V ∣ ∣ = ∣ ∣ U A ∣ ∣ \begin{align*}
\vert \vert A \vert \vert = \vert \vert A V \vert \vert = \vert \vert U A \vert \vert
\end{align*} ∣∣ A ∣∣ = ∣∣ A V ∣∣ = ∣∣ U A ∣∣ where V V V U U U
Hence, these norms are all also unitarily invariant, meaning ∣ ∣ A ∣ ∣ = ∣ ∣ U A U † ∣ ∣ \vert \vert A \vert \vert = \vert \vert UAU^{\dagger} \vert \vert ∣∣ A ∣∣ = ∣∣ U A U † ∣∣
The singular valued decomposition of a matrix A ∈ M n m ( C ) A~\in~\mathbb{M}_{nm}(\mathbb{C}) A ∈ M nm ( C )
A = U D V † , A = U D V^{\dagger}, A = U D V † , where U U U m × m m \times m m × m V V V n × n n \times n n × n D D D m × n m \times n m × n A A A
Consider a norm ∣ ∣ ⋅ ∣ ∣ : M n m ( C ) → R + \vert \vert \cdot \vert \vert: \mathbb{M}_{nm}(\mathbb{C}) \rightarrow \mathbb{R}^+ ∣∣ ⋅ ∣∣ : M nm ( C ) → R + W W W
A W = U D V † W , = U D V ~ † , \begin{align*}
AW &= UDV^\dagger W, \\
&= UD\tilde{V}^\dagger,
\end{align*} A W = U D V † W , = U D V ~ † , where V ~ † = V † W \tilde{V}^\dagger = V^\dagger W V ~ † = V † W A A A A W AW A W D D D ∣ ∣ A W ∣ ∣ = ∣ ∣ A ∣ ∣ \vert \vert A W \vert \vert = \vert \vert A \vert \vert ∣∣ A W ∣∣ = ∣∣ A ∣∣
The same argument can be made to prove that ∣ ∣ W A ∣ ∣ = ∣ ∣ A ∣ ∣ \vert \vert W A \vert \vert = \vert \vert A \vert \vert ∣∣ W A ∣∣ = ∣∣ A ∣∣
A matrix A ∈ M n n ( C ) A \in \mathbb{M}_{nn}(\mathbb{C}) A ∈ M nn ( C ) eigen-decomposition if
A = Q P Q † \begin{align*}
A = Q P Q^\dagger
\end{align*} A = QP Q † where Q Q Q A A A P P P A A A
All matrices have a singular valued decompostion, but not all matrices have an eigen-decomposition. Moreover, unlike the singular values, the eigenvalues can be negative and complex.
For Hermitian
Note, for Hermitian
Let A ∈ M A~\in~M A ∈ M A ∈ M m n ( F ) A~\in~\mathbb{M}_{mn}(\mathbb{F}) A ∈ M mn ( F )
∣ ∣ A ∣ ∣ tr = ∑ i μ i ( A ) , \vert \vert A \vert \vert_{\textrm{tr}} = \sum_{i} \mu_{i}(A), ∣∣ A ∣ ∣ tr = i ∑ μ i ( A ) , where μ i ( A ) \mu_{i}(A) μ i ( A ) A A A A A A
It is also given by
∣ ∣ A ∣ ∣ tr = tr ( A † A ) . \vert \vert A \vert \vert_{\textrm{tr}} = \textrm{tr} \big( \sqrt{A^{\dagger}A} \big). ∣∣ A ∣ ∣ tr = tr ( A † A ) . Let A ∈ M A~\in~M A ∈ M A ∈ M m n ( F ) A~\in~\mathbb{M}_{mn}(\mathbb{F}) A ∈ M mn ( F )
∣ ∣ A ∣ ∣ F = ∑ j n ∑ i m ∣ a i j ∣ 2 . \vert \vert A \vert \vert_{F} = \sqrt{ \sum_{j}^{n} \sum_{i}^{m} \vert a_{ij} \vert ^{2} }. ∣∣ A ∣ ∣ F = j ∑ n i ∑ m ∣ a ij ∣ 2 . It is also given by
∣ ∣ A ∣ ∣ F = tr ( A † A ) . \vert \vert A \vert \vert_{F} = \sqrt{\textrm{tr}(A^{\dagger}A)}. ∣∣ A ∣ ∣ F = tr ( A † A ) . Let A ∈ M A~\in~M A ∈ M A ∈ M m n ( F ) A~\in~\mathbb{M}_{mn}(\mathbb{F}) A ∈ M mn ( F ) ψ ∈ V \psi~\in~V ψ ∈ V ψ ∈ M m n ( F ) \psi~\in~\mathbb{M}_{mn}(\mathbb{F}) ψ ∈ M mn ( F ) ( ψ 1 , ψ 2 , … , ψ n ) (\psi_1, \psi_2, \ldots, \psi_n) ( ψ 1 , ψ 2 , … , ψ n )
The l 2 l_2 l 2
∣ ∣ ψ ∣ ∣ 2 = ( ∣ ψ 1 ∣ 2 + ∣ ψ 2 ∣ 2 + … + ∣ ψ n ∣ 2 ) 1 / 2 \begin{align*}
\vert \vert \psi \vert \vert_{2} &= \big( \vert \psi_1 \vert^2 + \vert \psi_2 \vert^2 + \ldots + \vert \psi_n \vert^2)^{1/2}
\end{align*} ∣∣ ψ ∣ ∣ 2 = ( ∣ ψ 1 ∣ 2 + ∣ ψ 2 ∣ 2 + … + ∣ ψ n ∣ 2 ) 1/2 The l 2 l_2 l 2 operator norm l 2 l_2 l 2
∣ ∣ A ∣ ∣ 2 = sup ψ ∣ ∣ A ψ ∣ ∣ 2 ∣ ∣ ψ ∣ ∣ 2 \vert \vert A \vert \vert_2 = \sup_{\psi} \frac{ \vert \vert A \psi \vert \vert_2 }{ \vert \vert \psi \vert \vert_{2}} ∣∣ A ∣ ∣ 2 = ψ sup ∣∣ ψ ∣ ∣ 2 ∣∣ A ψ ∣ ∣ 2 It is also given by
∣ ∣ A ∣ ∣ 2 = μ max ( A † A ) \vert \vert A \vert \vert_2 = \sqrt{\mu_{\textrm{max}}(A^{\dagger}A)} ∣∣ A ∣ ∣ 2 = μ max ( A † A ) Let A ∈ M A~\in~M A ∈ M A ∈ M m n ( F ) A~\in~\mathbb{M}_{mn}(\mathbb{F}) A ∈ M mn ( F ) k k k
∣ ∣ A ∣ ∣ ∗ k = ∑ i k ∣ μ i ↓ ( A ) ∣ , \vert \vert A \vert \vert^{k} _{*} = \sum_{i}^{k} \vert \mu^{\downarrow}_{i}(A) \vert, ∣∣ A ∣ ∣ ∗ k = i ∑ k ∣ μ i ↓ ( A ) ∣ , where μ i ↓ ( A ) \mu^{\downarrow}_i (A) μ i ↓ ( A ) i i i A A A μ i ↓ ( A ) ≥ μ i + 1 ↓ ( A ) ∀ i \mu^{\downarrow}_i(A) \geq \mu^{\downarrow}_{i+1}(A)~\forall~i μ i ↓ ( A ) ≥ μ i + 1 ↓ ( A ) ∀ i k k k
Ky Fan Dominance : Let B ∈ M B~\in~M B ∈ M B ∈ M m n ( F ) B~\in~\mathbb{M}_{mn}(\mathbb{F}) B ∈ M mn ( F )
∣ ∣ A ∣ ∣ ∗ k ≤ ∣ ∣ B ∣ ∣ ∗ k , \vert \vert A \vert \vert^{k} _{*} \leq \vert \vert B \vert \vert^{k} _{*}, ∣∣ A ∣ ∣ ∗ k ≤ ∣∣ B ∣ ∣ ∗ k , for all k k k
∣ ∣ A ∣ ∣ ≤ ∣ ∣ B ∣ ∣ . \vert \vert A \vert \vert \leq \vert \vert B \vert \vert. ∣∣ A ∣∣ ≤ ∣∣ B ∣∣. Let A ∈ M A~\in~M A ∈ M A ∈ M m n ( F ) A~\in~\mathbb{M}_{mn}(\mathbb{F}) A ∈ M mn ( F )
∣ ∣ A ∣ ∣ p = ( ∑ i k [ μ i ( A ) ] p ) 1 p , \vert \vert A \vert \vert_{p} = \bigg( \sum_{i}^{k} \big[\mu_{i}(A)\big]^{p} \bigg)^{\frac{1}{p}}, ∣∣ A ∣ ∣ p = ( i ∑ k [ μ i ( A ) ] p ) p 1 , where μ i ( A ) \mu_i(A) μ i ( A ) A A A
Equally, Schatten norms can be written as
∣ ∣ A ∣ ∣ p = tr [ ∣ A ∣ p ] 1 p , \vert \vert A \vert \vert_{p} = \textrm{tr} \big[ \vert A \vert ^{p} \big]^{\frac{1}{p}}, ∣∣ A ∣ ∣ p = tr [ ∣ A ∣ p ] p 1 , where ∣ A ∣ = A A † \vert A \vert = \sqrt{AA^{\dagger}} ∣ A ∣ = A A †
If p = 1 p=1 p = 1 trace norm p = 2 p=2 p = 2 Frobenius Norm p = ∞ p=\infty p = ∞ l 2 l_{2} l 2
The following norms do not necessarily depend on the singular values and hence the above conditions might not hold.
This is an induced norm.
Let A ∈ M A~\in~M A ∈ M A ∈ F m × n A~\in~\mathbb{F}^{m \times n} A ∈ F m × n A A A A : V → W A: V \rightarrow W A : V → W V V V F m \mathbb{F}^{m} F m W W W F n \mathbb{F}^{n} F n
∣ ∣ A ∣ ∣ o p = inf { c ≥ 0 : ∣ ∣ A ψ ∣ ∣ W ≤ c ∣ ∣ ψ ∣ ∣ V ∀ ψ ∈ V } , \vert \vert A \vert \vert_{op} = \inf \{c \geq 0: \vert \vert A \psi \vert \vert_{W} \leq c \vert \vert \psi \vert \vert_{V}~\forall~\psi~\in~V \}, ∣∣ A ∣ ∣ o p = inf { c ≥ 0 : ∣∣ A ψ ∣ ∣ W ≤ c ∣∣ ψ ∣ ∣ V ∀ ψ ∈ V } , where ∣ ∣ ⋅ ∣ ∣ W \vert \vert \cdot \vert \vert_{W} ∣∣ ⋅ ∣ ∣ W ∣ ∣ ⋅ ∣ ∣ V \vert \vert \cdot \vert \vert_{V} ∣∣ ⋅ ∣ ∣ V V V V W W W
The operator norm can be colloquially thought of as the maximum amount a map A A A V V V
Let A ∈ M A~\in~M A ∈ M A ∈ F m × n A~\in~\mathbb{F}^{m \times n} A ∈ F m × n L p , q L_{p,q} L p , q p , q ≥ 1 p,q \geq 1 p , q ≥ 1
∣ ∣ A ∣ ∣ p , q = ( ∑ j n ( ∑ i m ∣ a i j ∣ p ) q p ) 1 q , \vert \vert A \vert \vert_{p,q} = \biggl( \sum^{n}_{j} \bigg( \sum_{i}^{m} \vert a_{ij} \vert^{p} \bigg)^{\frac{q}{p}} \biggl)^{\frac{1}{q}}, ∣∣ A ∣ ∣ p , q = ( j ∑ n ( i ∑ m ∣ a ij ∣ p ) p q ) q 1 , where a i j a_{ij} a ij A A A
The L 2 , 1 L_{2,1} L 2 , 1 l 2 l_{2} l 2 A A A
∣ ∣ A ∣ ∣ 2 , 1 = ∑ j n ∣ ∣ a j ∣ ∣ 2 = ∑ j n ( ∑ i m ∣ a i j ∣ 2 ) 1 2 \vert \vert A \vert \vert_{2,1} = \sum^{n}_{j} \vert \vert a_j \vert \vert_{2} = \sum^{n}_{j} \bigg( \sum_{i}^{m} \vert a_{ij} \vert^{2} \bigg)^{\frac{1}{2}} ∣∣ A ∣ ∣ 2 , 1 = j ∑ n ∣∣ a j ∣ ∣ 2 = j ∑ n ( i ∑ m ∣ a ij ∣ 2 ) 2 1 where a j a_{j} a j A A A
Cross Norms ¶ Consider a tensor product space V = X ⊗ Y V = \mathcal{X} \otimes \mathcal{Y} V = X ⊗ Y
A norm, ∣ ∣ ⋅ ∣ ∣ \vert \vert \cdot \vert \vert ∣∣ ⋅ ∣∣
∣ ∣ V ∣ ∣ = ∣ ∣ X ⊗ Y ∣ ∣ = ∣ ∣ X ∣ ∣ ∣ ∣ Y ∣ ∣ . \vert \vert V \vert \vert = \vert \vert \mathcal{X} \otimes \mathcal{Y} \vert \vert = \vert \vert \mathcal{X} \vert \vert ~ \vert \vert \mathcal{Y} \vert \vert. ∣∣ V ∣∣ = ∣∣ X ⊗ Y ∣∣ = ∣∣ X ∣∣ ∣∣ Y ∣∣. All norms that depend only on the singular values are cross norms.
Note : this is a somewhat hand-wavey proof.
Let the singular valued decomposition of a matrix A ∈ M n m ( C ) A~\in~\mathbb{M}_{nm}(\mathbb{C}) A ∈ M nm ( C )
A = U D A V † , A = U D_A V^{\dagger}, A = U D A V † , and the singular valued decomposition of a matrix B ∈ M n m ( C ) B~\in~\mathbb{M}_{nm}(\mathbb{C}) B ∈ M nm ( C )
B = W D B T † . B = W D_B T^{\dagger}. B = W D B T † . Hence, the singular values of A ⊗ B A \otimes B A ⊗ B A A A B B B
The singular valued decomposition of a matrix A ⊗ B A \otimes B A ⊗ B
A ⊗ B = ( U ⊗ W ) ( D A ⊗ D B ) ( V ⊗ T ) † . \begin{align*}
A \otimes B = (U \otimes W)(D_A \otimes D_B)(V \otimes T)^\dagger.
\end{align*} A ⊗ B = ( U ⊗ W ) ( D A ⊗ D B ) ( V ⊗ T ) † . By observing this fact, it can be seen that any norms that depends only the singular values of a matrix can be factorised into a product of the norm applied to A A A B B B
Useful Norm Inequalities ¶
Let H \mathcal{H} H Hilbert space A , B ∈ H A, B \in \mathcal{H} A , B ∈ H
The following inequality:
∣ ∣ A B ∣ ∣ r ≤ ∣ ∣ A ∣ ∣ p ∣ ∣ B ∣ ∣ q , \vert \vert AB \vert \vert_r \leq \vert \vert A \vert \vert_p ~ \vert \vert B \vert \vert_q, ∣∣ A B ∣ ∣ r ≤ ∣∣ A ∣ ∣ p ∣∣ B ∣ ∣ q , where 1 p + 1 q = 1 r \frac{1}{p} + \frac{1}{q} = \frac{1}{r} p 1 + q 1 = r 1
Note, Hölder’s Inequality is more general than this, and applies to vector spaces other than Hilbert spaces and objects other than operators. See here for some details
Let A ∈ H 1 ⊗ H 2 A \in \mathcal{H}_1 \otimes \mathcal{H}_2 A ∈ H 1 ⊗ H 2 H \mathcal{H} H ∣ ∣ ⋅ ∣ ∣ p \vert \vert \cdot \vert \vert_p ∣∣ ⋅ ∣ ∣ p Schatten norm
∣ ∣ tr 1 [ A ] ∣ ∣ p ≤ [ d i m ( H 1 ) ] ( p − 1 ) / p ∣ ∣ A ∣ ∣ p . \vert \vert ~ \textrm{tr}_1 [ A ] ~ \vert \vert_p \leq \big[ {\rm dim}(\mathcal{H}_1)]^{(p-1)/p} \vert \vert A \vert \vert_p. ∣∣ tr 1 [ A ] ∣ ∣ p ≤ [ dim ( H 1 ) ] ( p − 1 ) / p ∣∣ A ∣ ∣ p . The result it more general than this and more details can be found here
Let x , y ∈ V \bm{x}, \bm{y}~\in~V x , y ∈ V V V V inner product space F \mathbb{F} F ( ⋅ , ⋅ ) → F 1 (\cdot, \cdot) \rightarrow \mathbb{F}^{1} ( ⋅ , ⋅ ) → F 1
∣ ( x , y ) ∣ 2 ≤ ∣ ∣ x ∣ ∣ ∣ ∣ y ∣ ∣ , \vert (\bm{x}, \bm{y}) \vert ^{2} \leq \vert \vert \bm{x} \vert \vert ~ \vert \vert \bm{y} \vert \vert, ∣ ( x , y ) ∣ 2 ≤ ∣∣ x ∣∣ ∣∣ y ∣∣ , where ∣ ⋅ ∣ \vert \cdot \vert ∣ ⋅ ∣
∣ ∣ x ∣ ∣ = ( x , x ) . \vert \vert \bm{x} \vert \vert = \sqrt{ (\bm{x}, \bm{x}) }. ∣∣ x ∣∣ = ( x , x ) .
Rastegin, A. E. (2012). Relations for certain symmetric norms and anti-norms before and after partial trace . 10.48550/ARXIV.1202.3853