 # Linear algebra

## Linear algebra Topics

Sort by:

### Infinitesimal matrix change

Let , , and be square matrices with small, and define(1)where is the identity matrix. Then the inverse of is approximately(2)This can be seen by multiplying(3)(4)(5)(6)Note that if we instead let , and look for an inverse of the form , we obtain(7)(8)(9)(10)In order to eliminate the term, we require . However, then , so so there can be no inverse of this form.The exact inverse of can be found as follows.(11)so(12)Using a general matrix inverse identity then gives(13)

### Matrix power

The power of a matrix for a nonnegative integer is defined as the matrix product of copies of ,A matrix to the zeroth power is defined to be the identity matrix of the same dimensions, . The matrix inverse is commonly denoted , which should not be interpreted to mean .

### Strassen formulas

The usual number of scalar operations (i.e., the total number of additions and multiplications) required to perform matrix multiplication is(1)(i.e., multiplications and additions). However, Strassen (1969) discovered how to multiply two matrices in(2)scalar operations, where is the logarithm to base 2, which is less than for . For a power of two (), the two parts of (2) can be written(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)so (◇) becomes(13)Two matrices can therefore be multiplied(14)(15)with only(16)scalar operations (as it turns out, seven of them are multiplications and 18 are additions). Define the seven products (involving a total of 10 additions) as(17)(18)(19)(20)(21)(22)(23)Then the matrix product is given using the remaining eight additions as(24)(25)(26)(27)(Strassen 1969, Press et al. 1989).Matrix inversion of a matrix to yield can also be done in fewer operations than expected using the formulas(28)(29)(30)(31)(32)(33)(34)(35)(36)(37)(38)(Strassen..

### Matrix multiplication

The product of two matrices and is defined as(1)where is summed over for all possible values of and and the notation above uses the Einstein summation convention. The implied summation over repeated indices without the presence of an explicit sum sign is called Einstein summation, and is commonly used in both matrix and tensor analysis. Therefore, in order for matrix multiplication to be defined, the dimensions of the matrices must satisfy(2)where denotes a matrix with rows and columns. Writing out the product explicitly,(3)where(4)(5)(6)(7)(8)(9)(10)(11)(12)Matrix multiplication is associative, as can be seenby taking(13)where Einstein summation is again used. Now, since , , and are scalars, use the associativity of scalar multiplication to write(14)Since this is true for all and , it must be true that(15)That is, matrix multiplication is associative. Equation(13) can therefore be written(16)without ambiguity. Due to associativity,..

### Hermitian part

Every complex matrix can be broken into a Hermitianpart(i.e., is a Hermitian matrix) and an antihermitian part(i.e., is an antihermitian matrix). Here, denotes the adjoint.

### Square root method

The square root method is an algorithm which solves the matrixequation(1)for , with a symmetric matrix and a given vector. Convert to a triangular matrix such that(2)where is the transpose. Then(3)(4)so(5)giving the equations(6)(7)(8)(9)(10)These give(11)(12)(13)(14)(15)giving from . Now solve for in terms of the s and ,(16)(17)(18)which gives(19)(20)(21)Finally, find from the s and ,(22)(23)(24)giving the desired solution,(25)(26)(27)

### Matrix inverse

The inverse of a square matrix , sometimes called a reciprocal matrix, is a matrix such that(1)where is the identity matrix. Courant and Hilbert (1989, p. 10) use the notation to denote the inverse matrix.A square matrix has an inverse iff the determinant (Lipschutz 1991, p. 45). The so-called invertible matrix theorem is major result in linear algebra which associates the existence of a matrix inverse with a number of other equivalent properties. A matrix possessing an inverse is called nonsingular, or invertible.The matrix inverse of a square matrix may be taken in the Wolfram Language using the function Inverse[m].For a matrix(2)the matrix inverse is(3)(4)For a matrix(5)the matrix inverse is(6)A general matrix can be inverted using methods such as the Gauss-Jordan elimination, Gaussian elimination, or LU decomposition.The inverse of a product of matrices and can be expressed in terms of and . Let(7)Then(8)and(9)Therefore,(10)so(11)where..

### Matrix exponential

The power series that defines the exponential map also defines a map between matrices. In particular,(1)(2)(3)converges for any square matrix , where is the identity matrix. The matrix exponential is implemented in the Wolfram Language as MatrixExp[m].The Kronecker sum satisfies the nice property(4)(Horn and Johnson 1994, p. 208).Matrix exponentials are important in the solution of systems of ordinary differential equations (e.g., Bellman 1970).In some cases, it is a simple matter to express the matrix exponential. For example, when is a diagonal matrix, exponentiation can be performed simply by exponentiating each of the diagonal elements. For example, given a diagonal matrix(5)The matrix exponential is given by(6)Since most matrices are diagonalizable,it is easiest to diagonalize the matrix before exponentiating it.When is a nilpotent matrix, the exponential is given by a matrix polynomial because some power of vanishes...

### Matrix equality

Two matrices and are said to be equal iff(1)for all . Therefore,(2)while(3)

### Matrix direct sum

The matrix direct sum of matrices constructs a block diagonal matrix from a set of square matrices, i.e.,(1)(2)

Denote the sum of two matrices and (of the same dimensions) by . The sum is defined by adding entries with the same indicesover all and . For example,Matrix addition is therefore both commutative andassociative.

### Kronecker sum

The Kronecker sum is the matrix sum defined by(1)where and are square matrices of order and , respectively, is the identity matrix of order , and denotes the Kronecker product.For example, the Kronecker sum of two matrices and is given by(2)The Kronecker sum satisfies the nice property(3)where denotes a matrix exponential.

### Antihermitian part

Every complex matrix can be broken into a Hermitian part(i.e., is a Hermitian matrix) and an antihermitian part(i.e., is an antihermitian matrix). Here, denotes the conjugate transpose.

### Natural norm

Let be a vector norm of a vector such thatThen is a matrix norm which is said to be the natural norm induced (or subordinate) to the vector norm . For any natural norm,where is the identity matrix. The natural matrix norms induced by the L1-norm, L2-norm, and L-infty-norm are called the maximum absolute column sum norm, spectral norm, and maximum absolute row sum norm, respectively.

### Maximum absolute row sum norm

The natural norm induced by the L-infty-normis called the maximum absolute row sum norm and is defined byfor a matrix . This matrix norm is implemented as Norm[m, Infinity].

### Refined alternating sign matrix conjecture

The numerators and denominators obtained by taking the ratios of adjacent terms in the triangular array of the number of "bordered" alternating sign matrices with a 1 at the top of column are, respectively, the numbers in the (2, 1)- and (1, 2)-Pascal triangles which are different from 1. This conjecture was proven by Zeilberger (1996).

### Alternating sign matrix

An alternating sign matrix is a matrix of 0s, 1s, and s in which the entries in each row or column sum to 1 and the nonzero entries in each row and column alternate in sign. The first few for , 2, ... are shown below:(1)(2)(3)(4)Such matrices satisfy the additional property that s in a row or column must have a "outside" it (i.e., all s are "bordered" by s). The numbers of alternating sign matrices for , 2, ... are given by 1, 2, 7, 42, 429, 7436, 218348, ... (OEIS A005130).The conjecture that the number of is explicitly given by the formula(5)now proven to be true, was known as the alternating sign matrix conjecture. can be expressed in closed form as a complicated function of Barnes G-functions, but additional simplification is likely possible.A recurrence relation for is given by(6)where is the gamma function.Let be the number of alternating sign matrices with one in the top row occurring in the th position. Then(7)The result(8)for..

### Generalized vandermonde matrix

A generalized Vandermonde matrix of two sequences and where is an increasing sequence of positive integers and is an increasing sequence of nonnegative integers of the same length is the outer product of and with multiplication operation given by the power function. The generalized Vandermonde matrix can be implemented in the Wolfram Language as Vandermonde[a_List?VectorQ, b_List?VectorQ] := Outer[Power, a, b] /; Equal @@ Length /@ {a, b}A generalized Vandermonde matrix is a minor of a Vandermonde matrix. Alternatively, it has the same form as a Vandermonde matrix , where is an increasing sequence of positive integers, except now is any increasing sequence of nonnegative integers. In the special case of a Vandermonde matrix, .While there is no general formula for the determinant of a generalized Vandermonde matrix, its determinant is always positive. Since any minor of a generalized Vandermonde matrix is also a generalized Vandermonde..

### Random matrix

A random matrix is a matrix of given type and size whoseentries consist of random numbers from some specified distribution.Random matrix theory is cited as one of the "modern tools" used in Catherine'sproof of an important result in prime number theory in the 2005 film Proof.For a real matrix with elements having a standard normal distribution, the expected number of real eigenvalues is given by(1)(2)where is a hypergeometric function and is a beta function (Edelman et al. 1994, Edelman and Kostlan 1994). has asymptotic behavior(3)Let be the probability that there are exactly real eigenvalues in the complex spectrum of the matrix. Edelman (1997) showed that(4)which is the smallest probability of all s. The entire probability function of the number of expected real eigenvalues in the spectrum of a Gaussian real random matrix was derived by Kanzieper and Akemann (2005) as(5)where(6)(7)In (6), the summation runs over all partitions..

### Gaussian elimination

Gaussian elimination is a method for solving matrixequations of the form(1)To perform Gaussian elimination starting with the system of equations(2)compose the "augmented matrix equation"(3)Here, the column vector in the variables is carried along for labeling the matrix rows. Now, perform elementary row operations to put the augmented matrix into the upper triangular form(4)Solve the equation of the th row for , then substitute back into the equation of the st row to obtain a solution for , etc., according to the formula(5)In the Wolfram Language, RowReduce performs a version of Gaussian elimination, with the equation being solved by GaussianElimination[m_?MatrixQ, v_?VectorQ] := Last /@ RowReduce[Flatten /@ Transpose[{m, v}]]LU decomposition of a matrix is frequently usedas part of a Gaussian elimination process for solving a matrix equation.A matrix that has undergone Gaussian elimination is said to be in echelonform.For..

### Unitary matrix

A square matrix is a unitary matrix if(1)where denotes the conjugate transpose and is the matrix inverse. For example,(2)is a unitary matrix.Unitary matrices leave the length of a complex vectorunchanged.For real matrices, unitary is the same as orthogonal. In fact, there are some similarities between orthogonal matrices and unitary matrices. The rows of a unitary matrix are a unitary basis. That is, each row has length one, and their Hermitian inner product is zero. Similarly, the columns are also a unitary basis. In fact, given any unitary basis, the matrix whose rows are that basis is a unitary matrix. It is automatically the case that the columns are another unitary basis.A matrix can be tested to see if it is unitary using the Wolfram Language function: UnitaryQ[m_List?MatrixQ] := ([email protected] @ m . m == IdentityMatrix @ Length @ m)The definition of a unitary matrix guarantees that(3)where is the identity matrix. In particular,..

### Permanent

The permanent is an analog of a determinant where all the signs in the expansion by minors are taken as positive. The permanent of a matrix is the coefficient of in(1)(Vardi 1991). Another equation is the Ryser formula(2)where the sum is over all subsets of , and is the number of elements in (Vardi 1991). Muir (1960, p. 19) uses the notation to denote a permanent.The permanent can be implemented in the WolframLanguage as Permanent[m_List] := With[{v = Array[x, Length[m]]}, Coefficient[Times @@ (m.v), Times @@ v] ]The computation of permanents has been studied fairly extensively in algebraic complexity theory. The complexity of the best-known algorithms grows as the exponent of the matrix size (Knuth 1998, p. 499), which would appear to be very surprising, given the permanent's similarity to the tractable determinant. Computation of the permanent is #P-complete (i.e, sharp-P complete; Valiant 1979).If is a unitary matrix, then(3)(Minc..

### Orthogonal matrix

A matrix is an orthogonal matrix if(1)where is the transpose of and is the identity matrix. In particular, an orthogonal matrix is always invertible, and(2)In component form,(3)This relation make orthogonal matrices particularly easy to compute with, since the transpose operation is much simpler than computing an inverse.For example,(4)(5)are orthogonal matrices. A matrix can be tested to see if it is orthogonal using the Wolfram Language code: OrthogonalMatrixQ[m_List?MatrixQ] := (Transpose[m].m == IdentityMatrix @ Length @ m)The rows of an orthogonal matrix are an orthonormal basis. That is, each row has length one, and are mutually perpendicular. Similarly, the columns are also an orthonormal basis. In fact, given any orthonormal basis, the matrix whose rows are that basis is an orthogonal matrix. It is automatically the case that the columns are another orthonormal basis.The orthogonal matrices are precisely those matrices..

### Normal matrix

A square matrix is a normal matrix ifwhere is the commutator and denotes the conjugate transpose. For example, the matrixis a normal matrix, but is not a Hermitian matrix. A matrix can be tested to see if it is normal using Wolfram Language function: NormalMatrixQ[a_List?MatrixQ] := Module[ {b = Conjugate @ Transpose @ a}, a. b === b. a ]Normal matrices arise, for example, from a normalequation.The normal matrices are the matrices which are unitarily diagonalizable, i.e., is a normal matrix iff there exists a unitary matrix such that is a diagonal matrix. All Hermitian matrices are normal but have real eigenvalues, whereas a general normal matrix has no such restriction on its eigenvalues. All normal matrices are diagonalizable, but not all diagonalizable matrices are normal.The following table gives the number of normal square matrices of given types for orders , 2, ....typeOEIScountsA0555472, 8, 68, 1124, ...A0555482, 12, 80, 2096, ...A0555493,..

### Symmetric matrix

A symmetric matrix is a square matrix that satisfies(1)where denotes the transpose, so . This also implies(2)where is the identity matrix. For example,(3)is a symmetric matrix. Hermitian matrices are a useful generalization of symmetric matrices for complex matricesA matrix can be tested to see if it is symmetric using the Wolfram Language code: SymmetricQ[m_List?MatrixQ] := (m === Transpose[m])Written explicitly, the elements of a symmetric matrix have the form(4)The symmetric part of any matrixmay be obtained from(5)A matrix is symmetric if it can be expressed in the form(6)where is an orthogonal matrix and is a diagonal matrix. This is equivalent to the matrix equation(7)which is equivalent to(8)for all , where . Therefore, the diagonal elements of are the eigenvalues of , and the columns of are the corresponding eigenvectors.The numbers of symmetric matrices of order on symbols are , , , , ..., . Therefore, for (0,1)-matrices, the..

### Minor

A minor is the reduced determinant of a determinant expansion that is formed by omitting the th row and th column of a matrix . So, for example, the minor of the above matrix is given byThe th minor can be computed in the Wolfram Language using Minor[m_List?MatrixQ, {i_Integer, j_Integer}] := Det[Drop[Transpose[Drop[Transpose[m], {j}]], {i}]]The Wolfram Language's built-in Minors[m] command instead gives the minors of a matrix obtained by deleting the st row and st column of , while Minors[m, k] gives the th minors of . The Minor code above therefore corresponds to th entry of MinorMatrix[m_List?MatrixQ] := Map[Reverse, Minors[m], {0, 1}]i.e., the definition Minors[m, i, j] is equivalent to MinorMatrix[m][[i, j]].

### Matrix minimal polynomial

The minimal polynomial of a matrix is the monic polynomial in of smallest degree such that(1)The minimal polynomial divides any polynomial with and, in particular, it divides the characteristic polynomial.If the characteristic polynomial factorsas(2)then its minimal polynomial is given by(3)for some positive integers , where the satisfy .For example, the characteristic polynomial of the zero matrix is , whiles its minimal polynomial is . However, the characteristic polynomial and minimal polynomial of(4)are both .The following Wolfram Language code will find the minimal polynomial for the square matrix in the variable . MatrixMinimalPolynomial[a_List?MatrixQ,x_]:=Module[ { i, n=1, qu={}, mnm={Flatten[IdentityMatrix[Length[a]]]} }, While[Length[qu]==0, AppendTo[mnm,Flatten[MatrixPower[a,n]]]; qu=NullSpace[Transpose[mnm]]; n++ ]; First[qu].Table[x^i,{i,0,n-1}] ]..

### Companion matrix

The companion matrix to a monic polynomial(1)is the square matrix(2)with ones on the subdiagonal and the last column given by the coefficients of . Note that in the literature, the companion matrix is sometimes defined with the rows and columns switched, i.e., the transpose of the above matrix.When is the standard basis, a companion matrix satisfies(3)for , as well as(4)including(5)The matrix minimal polynomial of the companion matrix is therefore , which is also its characteristic polynomial.Companion matrices are used to write a matrix in rational canonical form. In fact, any matrix whose matrix minimal polynomial has polynomial degree is similar to the companion matrix for . The rational canonical form is more interesting when the degree of is less than .The following Wolfram Language command gives the companion matrix for a polynomial in the variable . CompanionMatrix[p_, x_] := Module[ {n, w = CoefficientList[p, x]}, w = -w/Last[w];..

### Jacobian

Given a set of equations in variables , ..., , written explicitly as(1)or more explicitly as(2)the Jacobian matrix, sometimes simply called "the Jacobian" (Simon and Blume 1994) is defined by(3)The determinant of is the Jacobian determinant (confusingly, often called "the Jacobian" as well) and is denoted(4)The Jacobian matrix and determinant can be computed in the WolframLanguage using JacobianMatrix[f_List?VectorQ, x_List] := Outer[D, f, x] /; Equal @@ (Dimensions /@ {f, x}) JacobianDeterminant[f_List?VectorQ, x_List] := Det[JacobianMatrix[f, x]] /; Equal @@ (Dimensions /@ {f, x})Taking the differential(5)shows that is the determinant of the matrix , and therefore gives the ratios of -dimensional volumes (contents) in and ,(6)It therefore appears, for example, in the changeof variables theorem.The concept of the Jacobian can also be applied to functions in more than variables. For example, considering..

### Special unitary matrix

A square matrix is a special unitary matrix if(1)where is the identity matrix and is the conjugate transpose matrix, and the determinant is(2)The first condition means that is a unitary matrix, and the second condition provides a restriction beyond a general unitary matrix, which may have determinant for any real number. For example,(3)is a special unitary matrix. A matrix can be tested to see if it is a special unitary matrix using the Wolfram Language function SpecialUnitaryQ[m_List?MatrixQ] := (Conjugate @ Transpose @ m . m == IdentityMatrix @ Length @ m&& Det[m] == 1)The special unitary matrices are closed under multiplication and the inverse operation, and therefore form a matrix group called the special unitary group .

### Block diagonal matrix

A block diagonal matrix, also called a diagonal block matrix, is a square diagonal matrix in which the diagonal elements are square matrices of any size (possibly even ), and the off-diagonal elements are 0. A block diagonal matrix is therefore a block matrix in which the blocks off the diagonal are the zero matrices, and the diagonal matrices are square.Block diagonal matrices can be constructed out of submatrices in the WolframLanguage using the following code snippet: BlockDiagonalMatrix[b : {__?MatrixQ}] := Module[{r, c, n = Length[b], i, j}, {r, c} = Transpose[Dimensions /@ b]; ArrayFlatten[ Table[If[i == j, b[[i]], ConstantArray[0, {r[[i]], c[[j]]}]], {i, n}, {j, n} ] ] ]

### Special orthogonal matrix

A square matrix is a special orthogonal matrix if(1)where is the identity matrix, and the determinant satisfies(2)The first condition means that is an orthogonal matrix, and the second restricts the determinant to (while a general orthogonal matrix may have determinant or ). For example,(3)is a special orthogonal matrix since(4)and its determinant is . A matrix can be tested to see if it is a special orthogonal matrix using the Wolfram Language code SpecialOrthogonalQ[m_List?MatrixQ] := (Transpose[m] . m == IdentityMatrix @ Length @ m&& Det[m] == 1)The special orthogonal matrices are closed under multiplication and the inverse operation, and therefore form a matrix group called the special orthogonal group .

### Householder matrix

Householder (1953) first considered the matrix that now bears his name in the first couple of pages of his book. A Householder matrix for a real vector can be implemented in the Wolfram Language as: HouseholderMatrix[v_?VectorQ] := IdentityMatrix[Length[v]] - 2 Transpose[{v}] . {v} / (v.v)Trefethen and Bau (1997) gave an incorrect version of the formula for complex . D. Laurie gave a correct version by interpreting reflection along a given direction not as(1)where(2)is the projection onto the hyperplane orthogonal to (since this is in general not a unitary transformation), but as(3)Lehoucq (1996) independently gave an interpretation that still uses the formula , but choosing to be unitary.

### Array

An array is a "list of lists" with the length of each level of list the same. The size (sometimes called the "shape") of a -dimensional array is then indicated as . The most common type of array encountered is the two-dimensional rectangular array having columns and rows. If , a square array results. Sometimes, the order of the elements in an array is significant (as in a matrix), whereas at other times, arrays which are equivalent modulo reflections (and rotations, in the case of a square array) are considered identical (as in a magic square or prime array).In the Wolfram Language, an array of depth is represented using nested lists, and can be generated using the command Array[a, i, j, ...]. Similarly, the dimensions of an array can be found using Dimensions[t], and the command ArrayQ[expr] tests if an expression is a full array. Taking for example t=Array[a,{2,2,2,3}]gives the depth-4 list {{{{a[1,1,1,1],a[1,1,1,2],a[1,1,1,3]},..

### Antisymmetric matrix

An antisymmetric matrix is a square matrix thatsatisfies the identity(1)where is the matrix transpose. For example,(2)is antisymmetric. Antisymmetric matrices are commonly called "skew symmetric matrices" by mathematicians.A matrix may be tested to see if it is antisymmetric using the Wolfram Language function AntisymmetricQ[m_List?MatrixQ] := (m === -Transpose[m])In component notation, this becomes(3)Letting , the requirement becomes(4)so an antisymmetric matrix must have zeros on its diagonal. The general antisymmetric matrix is of the form(5)Applying to both sides of the antisymmetry condition gives(6)Any square matrix can be expressed as the sum of symmetric and antisymmetric parts. Write(7)But(8)(9)so(10)which is symmetric, and(11)which is antisymmetric.All antisymmetric matrices of odd dimension are singular. This follows from the fact that(12)So, by the properties of determinants,(13)(14)Therefore,..

### Hermitian matrix

A square matrix is called Hermitian if it is self-adjoint. Therefore, a Hermitian matrix is defined as one for which(1)where denotes the conjugate transpose. This is equivalent to the condition(2)where denotes the complex conjugate. As a result of this definition, the diagonal elements of a Hermitian matrix are real numbers (since ), while other elements may be complex.Examples of Hermitian matrices include(3)and the Pauli matrices(4)(5)(6)Examples of Hermitian matrices include(7)An integer or real matrix is Hermitian iff it is symmetric. A matrix can be tested to see if it is Hermitian using the Wolfram Language function HermitianQ[m_List?MatrixQ] := (m === [email protected]@m)Hermitian matrices have real eigenvalues whose eigenvectors form a unitary basis. For real matrices, Hermitian is the same as symmetric.Any matrix which is not Hermitian can be expressed as the sum of a Hermitian matrix and a antihermitian matrix using(8)Let..

### Antihermitian matrix

A square matrix is antihermitian if it satisfies(1)where is the adjoint. For example, the matrix(2)is an antihermitian matrix. Antihermitian matrices are often called "skew Hermitian matrices" by mathematicians.A matrix can be tested to see if it is antihermitian using the Wolfram Language function AntihermitianQ[m_List?MatrixQ] := (m === -Conjugate[Transpose[m]])The set of antihermitian matrices is a vector space, and the commutator(3)of two antihermitian matrices is antihermitian. Hence, the antihermitian matrices are a Lie algebra, which is related to the Lie group of unitary matrices. In particular, suppose is a path of unitary matrices through , i.e.,(4)for all , where is the adjoint and is the identity matrix. The derivative at of both sides must be equal so(5)That is, the derivative of at the identity must be antihermitian.The matrix exponential map of an antihermitianmatrix is a unitary matrix...

### Piecewise linear function

A piecewise linear function is a function composed of some number of linear segments defined over an equal number of intervals, usually of equal size.For example, consider the function over the interval . If is approximated by a piecewise linear function over an increasing number of segments, e.g., 1, 2, 4, and 8, the accuracy of the approximation is seen to improve as the number of segments increases.In the first case, with a single segment, if we compute the Lagrangeinterpolating polynomial, the equation of the linear function results.The trapezoidal rule for numeric integrationis described in a similar manner.Piecewise linear functions are also key to some constructive derivations. The length of a "piece" is given by the(1)summing the length of a number of pieces gives(2)and taking the limit as , the sum becomes(3)which is simplify the usual arc length...

### Vector space projection

If is a -dimensional subspace of a vector space with inner product , then it is possible to project vectors from to . The most familiar projection is when is the x-axis in the plane. In this case, is the projection. This projection is an orthogonal projection.If the subspace has an orthonormal basis thenis the orthogonal projection onto . Any vector can be written uniquely as , where and is in the orthogonal subspace .A projection is always a linear transformation and can be represented by a projection matrix. In addition, for any projection, there is an inner product for which it is an orthogonal projection.

### Invertible linear map

An invertible linear transformation is a map between vector spaces and with an inverse map which is also a linear transformation. When is given by matrix multiplication, i.e., , then is invertible iff is a nonsingular matrix. Note that the dimensions of and must be the same.

### Vector space orientation

An ordered vector basis for a finite-dimensional vector space defines an orientation. Another basis gives the same orientation if the matrix has a positive determinant, in which case the basis is called oriented.Any vector space has two possible orientations since the determinant of an nonsingular matrix is either positive or negative. For example, in , is one orientation and is the other orientation. In three dimensions, the cross product uses the right-hand rule by convention, reflecting the use of the canonical orientation as .An orientation can be given by a nonzero element in the top exterior power of , i.e., . For example, gives the canonical orientation on and gives the other orientation.Some special vector space structures imply an orientation. For example, if is a symplectic form on , of dimension , then gives an orientation. Also, if is a complex vector space, then as a real vector space of dimension , the complex structure gives an orientation...

### Orthogonal set

A subset of a vector space , with the inner product , is called orthogonal if when . That is, the vectors are mutually perpendicular.Note that there is no restriction on the lengths of the vectors. If the vectors inan orthogonal set all have length one, then they are orthonormal.The notion of orthogonal makes sense for an abstract vector space over any field as long as there is a symmetric quadratic form. The usual orthogonal sets and groups in Euclidean space can be generalized, with applications to special relativity, differential geometry, and abstract algebra.

### Hermitian inner product

A Hermitian inner product on a complex vector space is a complex-valued bilinear form on which is antilinear in the second slot, and is positive definite. That is, it satisfies the following properties, where denotes the complex conjugate of . 1. 2. 3. 4. 5. 6. , with equality only if The basic example is the form(1)on , where and . Note that by writing , it is possible to consider , in which case is the Euclidean inner product and is a nondegenerate alternating bilinear form, i.e., a symplectic form. Explicitly, in , the standard Hermitian form is expressed below.(2)A generic Hermitian inner product has its real part symmetric positive definite, and its imaginary part symplectic by properties 5 and 6. A matrix defines an antilinear form, satisfying 1-5, by iff is a Hermitian matrix. It is positive definite (satisfying 6) when is a positive definite matrix. In matrix form,(3)and the canonical Hermitian inner product is when is the identity matrix...

### Vector space flag

An ascending chain of subspaces of a vector space. If is an -dimensional vector space, a flag of is a filtration(1)where all inclusions are strict. Hence(2)so that . If equality holds, then for all , and the flag is called complete or full. In this case it is a composition series of .A full flag can be constructed by fixing a basis of , and then taking for all .A flag of any length can be obtained from a full flag by taking out some of the subspaces. Conversely, every flag can be completed to a full flag by inserting suitable subspaces. In general, this can be done in different ways. The following flag of (3)can be completed by switching in any line of the -plane passing through the origin. Two different full flags are, for example,(4)and(5)Schubert varieties are projective varieties definedfrom flags...

### Orthogonal complement

The orthogonal complement of a subspace of the vector space is the set of vectors which are orthogonal to all elements of . For example, the orthogonal complement of the space generated by two non proportional vectors , of the real space is the subspace formed by all normal vectors to the plane spanned by and .In general, any subspace of an inner product space has an orthogonal complement andThis property extends to any subspace of a space equipped with a symmetric or differential -form or a Hermitian form which is nonsingular on .

### Haar condition

A set of vectors in Euclidean -space is said to satisfy the Haar condition if every set of vectors is linearly independent (Cheney 1999). Expressed otherwise, each selection of vectors from such a set is a basis for -space. A system of functions satisfying the Haar condition is sometimes termed a Tchebycheff system (Cheney 1999).

### Fundamental theorem of linear algebra

Given an matrix , the fundamental theorem of linear algebra is a collection of results relating various properties of the four fundamental matrix subspaces of . In particular: 1. and where here, denotes the range or column space of , denotes its transpose, and denotes its null space. 2. The null space is orthogonal to the row space . 1. There exist orthonormal bases for both the column space and the row space of . 4. With respect to the orthonormal bases of and , is diagonal. The third item on this list stems from Gram-Schmidt Orthonormalization; the fourth item stems from the singular value decomposition of . Also, while different, the first item is reminiscent of the rank-nullity theorem.The above figure summarizes some of the interactions between the four fundamental matrix subspaces for a real matrix including whether the spaces in question are subspaces of or , which subspaces are orthogonal to one another, and how the matrix maps various vectors..

### Lorentzian inner product

The standard Lorentzian inner product on is given by(1)i.e., for vectors and ,(2) endowed with the metric tensor induced by the above Lorentzian inner product is known as Minkowski space and is denoted .The Lorentzian inner product on is nothing more than a specific case of the more general Lorentzian inner product on -dimensional Lorentzian space with metric signature : In this more general environment, the inner product of two vectors and has the form(3)The Lorentzian inner product of two such vectors is sometimes denoted to avoid the possible confusion of the angled brackets with the standard Euclidean inner product (Ratcliffe 2006). Analogous presentations can be made if the equivalent metric signature (i.e., for Minkowski space) is used.The four-dimensional Lorentzian inner product is used as a tool in special relativity, namely as a measurement which is independent of reference frame and which replaces the typical Euclidean notion..

### Fundamental matrix subspaces

Given a real matrix , there are four associated vector subspaces which are known colloquially as its fundamental subspaces, namely the column spaces and the null spaces of the matrices and its transpose . These four subspaces are important for a number of reasons, one of which is the crucial role they play in the so-called fundamental theorem of linear algebra.The above figure summarizes some of the interactions between the four fundamental matrix subspaces for a real matrix including whether the spaces in question are subspaces of or , which subspaces are orthogonal to one another, and how the matrix maps various vectors relative to the subspace in which lies.In the event that , all four of the fundamental matrix subspaces are lines in . In this case, one can write for some vectors , whereby the directions of the four lines correspond to , , , and . An elementary fact from linear algebra is that these directions are also represented by the eigenvectors..

### Linearly independent

Two or more functions, equations, or vectors , , ..., which are not linearly dependent, i.e., cannot be expressed in the formwith , , ... constants which are not all zero are said to be linearly independent.A set of vectors , , ..., is linearly independent iff the matrix rank of the matrix is , in which case is diagonalizable.

### Fredholm's theorem

Fredholm's theorem states that, if is an matrix, then the orthogonal complement of the row space of is the null space of , and the orthogonal complement of the column space of is the null space of ,(1)(2)

### Orthonormal basis

A subset of a vector space , with the inner product , is called orthonormal if when . That is, the vectors are mutually perpendicular. Moreover, they are all required to have length one: .An orthonormal set must be linearly independent, and so it is a vector basis for the space it spans. Such a basis is called an orthonormal basis.The simplest example of an orthonormal basis is the standard basis for Euclidean space . The vector is the vector with all 0s except for a 1 in the th coordinate. For example, . A rotation (or flip) through the origin will send an orthonormal set to another orthonormal set. In fact, given any orthonormal basis, there is a rotation, or rotation combined with a flip, which will send the orthonormal basis to the standard basis. These are precisely the transformations which preserve the inner product, and are called orthogonal transformations.Usually when one needs a basis to do calculations, it is convenient to use an orthonormal..

### Linear combination

A sum of the elements from some set with constant coefficients placed in front of each. For example, a linear combination of the vectors , , and is given bywhere , , and are constants.

### Alternating multilinear form

An alternating multilinear form on a real vector space is a multilinear form(1)such that(2)for any index . For example,(3)is an alternating form on .An alternating multilinear form is defined on a module in a similar way, by replacing with the ring.

### Jacobi's theorem

Let be an -rowed minor of the th order determinant associated with an matrix in which the rows , , ..., are represented with columns , , ..., . Define the complementary minor to as the -rowed minor obtained from by deleting all the rows and columns associated with and the signed complementary minor to to be(1)Let the matrix of cofactors be given by(2)with and the corresponding -rowed minors of and , then it is true that(3)

### Determinant theorem

Given a square matrix , the following are equivalent: 1. . 2. The columns of are linearly independent. 3. The rows of are linearly independent. 4. Range() = . 5. Null() = . 6. has a matrix inverse.

### Jacobi's determinant identity

Let(1)(2)where and are matrices. Then(3)The proof follows from equating determinants on the two sides of the block matrices(4)where is the identity matrix and is the zero matrix.

### Determinant identities

A useful determinant identity allows the following determinant to be expressed using vector operations,(1)Additional interesting determinant identities include(2)(Muir 1960, p. 39),(3)(Muir 1960, p. 41),(4)(Muir 1960, p. 42),(5)(Muir 1960, p. 47),(6)(Muir 1960, p. 42),(7)(Muir 1960, p. 44), and the Cayley-Mengerdeterminant(8)(Muir 1960, p. 46), which is closely related to Heron'sformula.