MATRIX DECOMPOSITION ALGORITHMS

RISHABH CHAUDHARY Bhagwan Parshuram Institute of Technology, Delhi, India

ANKIT SABLOK Bhagwan Parshuram Institute of Technology, Delhi, India

DEEPAK GUPTA Bhagwan Parshuram Institute of Technology, Delhi, India

ABSTRACT Due to the rapid advancement in the field of Mathematics and Computation in the last two decades various techniques have surfaced solving some of the most astounding problems in Mathematics leading to fruitful results such as decrease in computation time. One such difficult problem that has had an enormous impact in the field of Mathematics and Computation is “The problem of factorizing the Matrices” so as to decrease the Computation time in some of the most elegant algorithms used in Computer Science and the branch of Operating Systems. Some of the techniques such as The Gaussian Elimination, The LUP Decompositions, The Singular Value Decompositions (SVD), The QR factorizations, Graham-Schmidt Decompositions and many others some of these techniques and their comparisons in various applications will be discussed in this paper and will unearth some of the various elegant laws of Matrix Algebra and Numerical Analysis. Index Terms — Gaussian Elimination, LUP Decompositions, Singular Value Decompositions (SVD), QR Decomposition, GrahamSchmidt Decompositions INTRODUCTION Matrix Decompositions are a very fundamental concept in linear algebra they are used for factorization of square or rectangular matrices. Besides factoring the matrix in different ways these decomposition schemes have their advantages and disadvantages our main interest here is to discuss a single but one fundamental kernel in dense linear algebra-: Matrix Decompositions. We consider performance effects of different decomposition algorithms applied to non-singular matrices (determinant not equal to 0) and show which outperforms which one. MATRIX DECOMPOSITION ALGORITHMS

A. Gaussian Elimination Gaussian elimination is one of the most efficient methods to perform the matrix decompositions or factorizations. In this method of factorization of a matrix the matrix is expressed as an upper triangular matrix which is obtained through the methods of forward substitution and back substitution and converted into an upper triangular matrix as it is easier to solve than the original matrix this is called as the technique of triangular triangulization this technique takes effectively 2n3/3 operations for an n x n matrix.

B. QR Decomposition In matrix decompositions a Q-R decomposition of a matrix is a decomposition of the matrix into an orthogonal (AAT=I) and an upper triangular matrix it is the basis gor all the eigenvalue algorithms. The Q-R decompositions are obtained using 3 methods GrahamSchmidt triangulizations, Householder triangulizations and Gives rotations in this paper we will only discuss the first two only.

C. Singular Value Decomposition (SVD) This is also one of the matrix decomposition schemes. This is also applicable for complex rectangular m x n. further details are in the subsequent sections below along with comparisons.

MATRIX DECOMPOSITION COMPARISON

matrices of the dimensions

A. Gaussian Elimination A matrix A=LU (where L is a unit lower triangular matrix and U is an upper triangular matrix) it is done by using the technique of triangular triangulizations which is done to reduce the matrix A into triangular systems which are easier to solve. The complexity of performing the LU decomposition of a matrix A which is n x n is 2n3/3 operations that is O (n3) .But gaussian elimination is not an efficient method for performing the LU decompositions because in Gaussian elimination without pivoting it sometimes the upper leftmost entry of the schur complement is sometimes 0 which is not possible for which we introduce the technique of pivoting. But it is the most efficient algorithm for positive –definite matrices. But if the matrix is sparse then the running time is 2n3 effectively O (n3). For tridiagonal sets, the procedures of LU decomposition, forward- and back substitution each take only O (N) operations and the whole solution can be encoded very concisely. B. QR Decomposition A matrix A=QR in linear algebra a QR decomposition or factorization of a matrix A is the decomposition of a matrix A into an orthogonal and right triangular matrix. The Q-R decompositions are generally used to solve the linear least squares problems. If the matrix A is non-singular then the Q-R factorization of A is a unique one if we want that the diagonal elements of R are positive. More generally we can factor a m x n rectangular matrix into an m x m unitary matrix and an m x n upper triangular matrix. There are 2 standard methods to evaluate Q-R factorization of A such as Graham-Schmidt triangulizations and Householder triangulizations but Householder triangulizations are much more efficient than Graham-Schmidt triangulizations because of the following statistics involved. The number of operations at the kth step are multiplications=2(n-k+1)2 , additions=(n-k+1)2 +(n-k+1)(n-k)+2 , division=1and square root=1. Summing all these operations over (n-1) steps we get the complexity as O (n3) but this involves much less computation time than the other triangulization methods. C. Singular Value Decomposition (SVD) In linear algebra the single value decomposition is an important factorization technique of a rectangular m x n matrix or a complex matrix it has many applications in the field of statistics and signal processing applications concerning SVD are pseudo inverse, least squares fitting , Matrix approximation. Suppose A is an m x n matrix whose entries come from a field K which may be a field of real numbers or complex numbers. Then there exists a factorization as A=U∑V* called the single value decomposition of the matrix A. where U is the m x m unitary matrix over the field K, the matrix ∑ is the m x n diagonal matrix with non negative real numbers on the diagonal and V* is the conjugate transpose of V. A non-negative real number σ is a singular value for M if and only if there exist unit-length vectors u in Km and v in Kn such that Mv=σu and M*u<σv where u and v are left singular and right singular vectors for σ respectively. The SVD of a matrix M is computed as two step procedure in the first step the matrix is reduced to a bidiagonal matrix this work takes about O(mn2) order time assuming that m≥n. The second step is to compute the SVD of a bidiagonal matrix and this is an iterative step and can be computed in O(n) order time. The first step can be done in 4mn2-(4n3/3) time assuming that only singular matrices are needed and not singular vectors otherwise if the matrix is very large then it can first be reduced to a triangular matrix using Q-R decompositions and then use Householder reflections to further reduce the matrix to bi-diagonal form and this can be done in 2mn2+2n3 time. Geometrically here the image of a unit sphere AxS has the geometry of an n dimensional hyperellipse. This decomposition scheme also makes use of the unitary matrix. The graph below shows the comparison of all the three decomposition algorithms:

Fig 1: Comparison of all the Matrix Decomposition CONCLUSION Seeing all the above comparisons we find that the LUP decompositions i.e Gaussian elimination with pivoting is the most efficient method of decomposition having the least computation cost of 2n3/3 floating point operations least computation cost of 2n3/3 floating point operations and hence used widely in many different decomposition problems in statistics and inverse finding. Volker Strassen computed the inverse of a matrix using the LUP decomposition in <5.64nlog27 time and the square matrix. REFERENCES [1] E. W. Weisstein. Matrix decomposition. MathWorld–A Wolfram Web Resource. [Online]. Available: http://mathworld.wolfram.com/MatrixDecomposition.html

[2] R. Bronson, Matrix Methods An Introduction, 2nd ed. Academic Press, 1991. [3] G. H.Golub and C. F. Loan, Matrix Computations, 2nd ed. Johns Hopkins Press, 1996. [4] L. Hubert, J. Meulman, and W. Heiser, “Two purposes for matrix factorization: A historical appraisal,” SIAM Review, vol. 42, no. 1, pp. 68–82, 2000. [5] P. Lancaster and M. Tismenestsky, The Theory of Matrices, 2nd ed., W. Rheinboldt, Ed. Academic Press, 1985. [6] M. T. Chu, R. E. Funderlic, and G. H. Golub, “A rank-one reduction formula and its applications to matrix factorizations,” SIAM Review, vol. 37, no. 4, pp. 512–530, 1995.17

MATRIX DECOMPOSITION ALGORITHMS A ... - PDFKUL.COM

[5] P. Lancaster and M. Tismenestsky, The Theory of Matrices, 2nd ed., W. Rheinboldt, Ed. Academic Press, 1985. [6] M. T. Chu, R. E. Funderlic, and G. H. Golub, ...

103KB Sizes 0 Downloads 257 Views

Recommend Documents

MATRIX DECOMPOSITION ALGORITHMS A ... - Semantic Scholar
... of A is a unique one if we want that the diagonal elements of R are positive. ... and then use Householder reflections to further reduce the matrix to bi-diagonal form and this can ... http://mathworld.wolfram.com/MatrixDecomposition.html ...

MATRIX DECOMPOSITION ALGORITHMS A ... - Semantic Scholar
solving some of the most astounding problems in Mathematics leading to .... Householder reflections to further reduce the matrix to bi-diagonal form and this can.

Non-Negative Matrix Factorization Algorithms ... - Semantic Scholar
Keywords—matrix factorization, blind source separation, multiplicative update rule, signal dependent noise, EMG, ... parameters defining the distribution, e.g., one related to. E(Dij), to be W C, and let the rest of the parameters in the .... contr

A new method of spectral decomposition of covariance matrix in mixed ...
covariance matrix in mixed effects models and its applications. WU Mixia & WANG Songgui. College of Applied Sciences, Beijing University of Technology, ...

Deterministic algorithms for skewed matrix products
Figure 1 A high-level pseudocode description of the algorithm. In ComputeSummary we iterate over the n outer products and to each one of them apply Lemma 1 such that only the b heaviest entries remain. We update the summary with the entries output by

Two recursive pivot-free algorithms for matrix inversion
The talk is organized as follows: Section 2 provides some necessary background ... with one-sided decomposition (H-inversion) with illustration and example.

Multivariate Coherence Decomposition: A Simulation ...
By calculating averaged coherence over all pairs of channels, we can know at which frequencies there are strong coherence. And then, by utilizing. MVCCDFD to corresponding frequencies we can get the 2D distributions of coherent sources at given frequ

A Nonparametric Variance Decomposition Using Panel Data
Oct 20, 2014 - In Austrian data, we find evidence that heterogeneity ...... analytical standard errors for our estimates without imposing functional forms on Fi, we.

Notes on Decomposition Methods - CiteSeerX
Feb 12, 2007 - Some recent reference on decomposition applied to networking problems ...... where di is the degree of net i, i.e., the number of subsystems ...

A Domain Decomposition Method based on the ...
Nov 1, 2007 - In this article a new approach is proposed for constructing a domain decomposition method based on the iterative operator splitting method.

Notes on Decomposition Methods - CiteSeerX
Feb 12, 2007 - matrix inversion lemma (see [BV04, App. C]). The core idea .... this trick is so simple that most people would not call it decomposition.) The basic ...

Notes on Decomposition Methods - CiteSeerX
Feb 12, 2007 - is adjacent to only two nodes, we call it a link. A link corresponds to a shared ..... exponential service time with rate cj. The conjugate of this ...

Convex Shape Decomposition
lem in shape related areas, such as computer vision, com- puter graphics and ... First, we give a mathematical definition of the decompo- sition. Definition 1.

Control of a DC matrix converter
May 2, 2002 - achieve the desired DC voltage for application to the load, such as the motor 19. ..... testing the sign of Vq in tests 117—119 of FIG. 8; i.e., d[3=—d[3 if ..... regeneration, such as for driving poWer tools, or in other applicatio

Intrinsic Image Decomposition Using a Sparse ...
A 3D plot of the WRBW coefficients (across RGB). updated using the computed detail coefficients stored .... used the “box” example from the MIT intrinsic image database [24]. This is shown in Fig. 4. We perform the. WRBW on both the original imag

A decomposition theorem for characteristic 0 henselian ...
Nov 6, 2007 - Swiss cheese. Definition. A swiss cheese is a subset of K of the form B\(C1 ∪...∪Cn), where B,C1,...,Cn are all balls (or K itself), with Ci ⊊ B.

A Study of Non-Smooth Convex Flow Decomposition
the components uc, us and ut of a divergence–free flow u = G⊥ψ, i.e., u = uc + us + ut ..... In Proceedings of the Conference on Computer ... H.D. Mittelmann.

A Structural Decomposition of the US Yield Curve
Tel.: 001 214 922 6715. ‡National Bank of Belgium ([email protected]). .... Et0/πt$&1( and a disturbance term 2Eb t. The structural parameters measure the.

A single route, full decomposition model of ...
mechanism models where only regular forms involve composition. .... This model of lexical storage and activation predicts that the precise form the root .... whom this response component was not identifiable in the grandaveraged file were not.

A Martingale Decomposition of Discrete Markov Chains
Apr 1, 2015 - Email: [email protected] ... may be labelled the efficient price and market microstructure noise, respectively. ... could be used to compare the long-run properties of the approximating Markov process with those of.

coordinatewise decomposition of group-valued borel functions
Fix γ0 ∈ Γ \ {1Γ}, define f : S → Γ by f(x, y) = ¨ γ0 ... the unique G-neighbor of z in Bn, and define recursively u : X → Γ,v : Y → Γ by ... Bi and ∃w ∈ Bn ((z, w) ∈ G).

Automated Problem Decomposition in Evolutionary ...
Automated Problem Decomposition in Evolutionary Algorithms. Sherri Goings. Department of Computer Science and Engineering. Michigan State University. 1.