Jos´e Antonio Apolin´ario Jr. (Editor)

RE VI EW

Military Institute of Engineering (IME) Rio de Janeiro, Brazil

UN DE R

QRD-RLS Adaptive Filtering

RE VI EW UN DE R

To Ana, Isabela, and Eduardo.

RE VI EW

Foreword

The foreword covers introductory remarks preceding the text of a book that are written by a person other than the author or editor of the book. If applicable, the foreword precedes the preface which is written by the author or editor of the book.

UN DE R

Cardiff, Wales, UK, August 2008

Prof. John G. McWhirter

iii

UN DE R RE VI EW

RE VI EW

Preface

The fast growth of the technological resources observed nowadays has triggered the development of new DSP techniques to cope with the requirements of modern industry. The research of efficient algorithms to be used in the ever increasing applications of adaptive filters has therefore developed tremendously. In such a scenario, the QRD-RLS-based algorithms are a good option in applications where speed of convergence is of paramount importance and an efficient, reliable and numerically robust adaptive filter is needed.

UN DE R

However, I believe that the nice features of this family of algorithms, in many occasions, are not used due simply to the fact that their matrix equations are not easy to understand. On the other hand, students, researchers and practitioners need to be constantly up-to-date with the recent developments, not only by attending conferences and reading journal papers, but also by referring to a comprehensive compendium, where all concepts were carefully matured and are presented in such a way as to provide easy understanding. This is the main goal of this book: to provide the reader with the necessary tools to understand and implement a variety of QRDRLS algorithms suitable to a vast number of applications.

This publication gathers some of the most recent developments as well as the basic concepts for a complete understanding of the QRD-RLS-based algorithms. Although this work does not cover all fronts of research in the field, it tries to bring together the most important topics for those who need an elegant and fast-converging adaptive filter. QR decomposition has been a pearl in applied mathematics for many years; its use in adaptive filtering is introduced in the first chapter of this book in the form of an annotated bibliography.

v

vi

Preface

RE VI EW

The fundamental chapters materialized from lecture notes of a short course given at the Helsinki University of Technology in the winter of 2004-2005, a number of conference and journal publications, and some theses I supervised. I was also lucky to receive contributions from many prominent authorities in the field.

This book consists of twelve chapters, going from fundamentals to more advanced aspects. Different algorithms are derived and presented, including basic, fast, lattice, multichannel and constrained versions. Important issues, such as numerical stability, performance in finite precision environments and VLSI oriented implementations are also addressed. All algorithms are derived using Givens rotations, although one chapter deals with implementations using Householder reflections. I hope the readers will find this book a handy guide to most aspects of theory and implementation details, quite useful in their professional practice.

Finally, I express my deep gratitude to all authors for their effort and competence in their timely and high quality contributions. I also thank the people from Springer, always very kind and professional. I am particularly grateful to my former DSc supervisor, Paulo S. R. Diniz, for his support and ability to motivate his pupils, and Marcello L. R. de Campos, the dear friend who, in the middle of a technical meeting on a sunny Friday, suggested this book.

UN DE R

Rio de Janeiro, Brazil September 2008

Jos´e A. Apolin´ario Jr. [email protected]

RE VI EW

Contents

QR Decomposition: an Annotated Bibliography . . . . . . . . . . . . . . . . . . Marcello L. R. de Campos and Gilbert Strang

1

2

Introduction to Adaptive Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jos´e A. Apolin´ario Jr. and Sergio L. Netto 2.1 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Error Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 The mean-square error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 The instantaneous square error . . . . . . . . . . . . . . . . . . . . . . 2.2.3 The weighted least-squares . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Adaptation Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 LMS and normalized-LMS algorithms . . . . . . . . . . . . . . . . 2.3.2 Data-reusing LMS algorithms . . . . . . . . . . . . . . . . . . . . . . . 2.3.3 RLS-type algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Computer Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Example 1: Misadjustment of the LMS algorithm . . . . . . . 2.4.2 Example 2: Convergence trajectories . . . . . . . . . . . . . . . . . 2.4.3 Example 3: Tracking performance . . . . . . . . . . . . . . . . . . . 2.4.4 Example 4: Algorithm stability . . . . . . . . . . . . . . . . . . . . . . 2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

UN DE R

1

3

4 9 9 10 10 12 12 16 22 24 25 26 26 29 31 31

Conventional and Inverse QRD-RLS Algorithms . . . . . . . . . . . . . . . . . 35 Jos´e Antonio Apolin´ario Jr. and Maria D. Miranda 3.1 The Least-Squares Problem and the QR Decomposition . . . . . . . . . 35 3.2 The Givens Rotation Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

vii

viii

Contents

The Conventional QRD-RLS Algorithm . . . . . . . . . . . . . . . . . . . . . . Initialization of the Triangularization Procedure . . . . . . . . . . . . . . . . On the Qθ (k) Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 The backward prediction problem . . . . . . . . . . . . . . . . . . . . 3.5.2 The forward prediction problem . . . . . . . . . . . . . . . . . . . . . 3.5.3 Interpreting the elements of Qθ (k) for a lower triangular Cholesky factor . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.4 Interpreting the elements of Qθ (k) for an upper triangular Cholesky factor . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 The Inverse QRD-RLS Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

60

62 63 64 66 67 69 72

Fast QRD-RLS Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Jos´e Antonio Apolin´ario Jr. and Paulo S. R. Diniz 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.2 Upper Triangularization Algorithms (Updating Forward Prediction Errors) . . . . . . . . . . . . . . . . . . . . . . . . 77 4.2.1 The FQR POS F Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.2.2 The FQR PRI F Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 80 4.3 Lower Triangularization Algorithms (Updating Backward Prediction Errors) . . . . . . . . . . . . . . . . . . . . . . . 82 4.3.1 The FQR POS B Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 83 4.3.2 The FQR PRI B Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.4 The Order Recursive Versions of the Fast QRD Algorithms . . . . . . 87 4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Appendix 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Appendix 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Appendix 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

UN DE R

4

45 49 52 54 57

RE VI EW

3.3 3.4 3.5

QRD Least-Squares Lattice Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 103 Jenq-Tay Yuan 5.1 Fundamentals of QRD-LSL Algorithms . . . . . . . . . . . . . . . . . . . . . . 104 5.2 LSL Interpolator and LSL Predictor . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Contents

ix

Multichannel Fast QRD-RLS Algorithms . . . . . . . . . . . . . . . . . . . . . . . 135 Ant´onio L. L. Ramos and Stefan Werner 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 6.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 6.2.1 Input vector for sequential-type multichannel algorithms . 140 6.2.2 Input vector for block-type multichannel algorithms . . . . 140 6.3 Sequential-type Multichannel Fast QRD-RLS Algorithms . . . . . . . 142 6.3.1 Triangularization of the information matrix . . . . . . . . . . . 142 6.3.2 A PRIORI and A POSTERIORI Versions . . . . . . . . . . . . . 146 6.3.3 Alternative implementations . . . . . . . . . . . . . . . . . . . . . . . . 148 6.4 Block-type Multichannel Fast QRD-RLS Algorithms . . . . . . . . . . . 151 6.4.1 The backward and forward prediction problems . . . . . . . . 151 6.4.2 A PRIORI and A POSTERIORI Versions . . . . . . . . . . . . . 155 6.4.3 Alternative implementations . . . . . . . . . . . . . . . . . . . . . . . . 159 6.5 Order-recursive Multichannel Fast QRD-RLS Algorithms . . . . . . . 161 6.6 Application Example and Computational Complexity Issues . . . . . 164 6.6.1 Application Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 6.6.2 Computational Complexity Issues . . . . . . . . . . . . . . . . . . . . 167 6.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

UN DE R

6

RE VI EW

5.2.1 LSL Interpolator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.2.2 Orthogonal Bases for LSL Interpolator . . . . . . . . . . . . . . . . 109 5.2.3 LSL Predictor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 5.3 SRF Givens Rotation with Feedback Mechanism . . . . . . . . . . . . . . . 111 5.4 SRF QRD-LSL Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 5.4.1 QRD Based on Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 114 5.4.2 SRF QRD-LSL Interpolation Algorithm . . . . . . . . . . . . . . 117 5.4.3 SRF QRD-LSL Prediction Algorithm and SRF Joint Process Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 5.5 SRF (QRD-LSL)-Based RLS Algorithm . . . . . . . . . . . . . . . . . . . . . . 127 5.6 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 5.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

7

Householder-Based RLS Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 Athanasios A. Rontogiannis and Sergios Theodoridis 7.1 Householder Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

x

Contents

Numerical Stability Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Phillip Regalia and Richard Le Borne 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 8.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 8.2.1 Conditioning, Forward Stability and Backward Stability . 198 8.3 The Conditioning of the Least Squares Problem . . . . . . . . . . . . . . . . 200 8.3.1 The Conditioning of the Least Squares Problem . . . . . . . . 201 8.3.2 Consistency, Stability and Convergence . . . . . . . . . . . . . . . 203 8.4 The Recursive QR Least Squares Methods . . . . . . . . . . . . . . . . . . . . 204 8.4.1 Full QR Decomposition Adaptive Algorithm . . . . . . . . . . . 204 8.5 Fast QR Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 8.5.1 Past Input Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 8.5.2 Reachable States in Fast Least-Squares Algorithms . . . . . 217 8.5.3 QR Decomposition Lattice Algorithm . . . . . . . . . . . . . . . . 220 8.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

UN DE R

8

RE VI EW

7.1.1 Hyperbolic Householder Transforms . . . . . . . . . . . . . . . . . 174 7.1.2 Row Householder Transforms . . . . . . . . . . . . . . . . . . . . . . . 175 7.2 The Householder RLS (HRLS) Algorithm . . . . . . . . . . . . . . . . . . . . 176 7.2.1 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 7.3 The Householder Block Exact QRD-RLS Algorithm . . . . . . . . . . . . 183 7.4 The Householder Block Exact Inverse QRD-RLS Algorithm . . . . . 186 7.5 Sliding window Householder Block Implementation . . . . . . . . . . . . 190 7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

9

Finite and Infinite Precision Properties of QRD-RLS Algorithms . . . 225 Paulo S. R. Diniz and Marcio G. Siqueira 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 9.2 Precision Analysis of the QR-Decomposition RLS Algorithm . . . . 226 9.2.1 Infinite Precision Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 227 9.2.2 Stability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 9.2.3 Error Propagation Analysis in Steady-State . . . . . . . . . . . . 235 9.2.4 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 9.3 Precision Analysis of the Fast QRD-Lattice Algorithm . . . . . . . . . . 247 9.3.1 Infinite Precision Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 248 9.3.2 Finite Precision Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

Contents

xi

RE VI EW

9.3.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 9.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 10 On Pipelined Implementations of QRD-RLS Adaptive Filters . . . . . . 259 Jun Ma and Keshab K. Parhi 10.1 QRD-RLS Systolic Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 10.2 The Annihilation-Reording Look-Ahead Technique . . . . . . . . . . . . . 264 10.2.1 Look-Ahead Through Block Processing . . . . . . . . . . . . . . . 264 10.2.2 Look-Ahead Through Iteration . . . . . . . . . . . . . . . . . . . . . . 266 10.2.3 Relationship with Multiply-Add Look-Ahead . . . . . . . . . . 268 10.2.4 Parallelism in Annihilation-Reording Look-Ahead . . . . . . 269 10.2.5 Pipelined and Block Processing Implementations . . . . . . . 271 10.2.6 Invariance of Bounded Input/Bounded Output . . . . . . . . . . 274 10.3 Pipelined CORDIC Based RLS Adaptive Filters . . . . . . . . . . . . . . . 274 10.3.1 Pipelined QRD-RLS with Implicit Weight Extraction . . . 274 10.3.2 Stability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 10.3.3 Pipelined QRD-RLS With Explicit Weight Extraction . . . 279 10.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Appendix 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

UN DE R

11 Weight Extraction of Fast QRD-RLS Algorithms . . . . . . . . . . . . . . . . . 291 Stefan Werner and Mohammed Mobien 11.1 FQRD-RLS Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292 11.1.1 QR decomposition algorithms . . . . . . . . . . . . . . . . . . . . . . . 292 11.1.2 FQR POS B algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 11.2 System Identification with FQRD-RLS . . . . . . . . . . . . . . . . . . . . . . . 295 11.2.1 Weight extraction in the FQRD-RLS algorithm . . . . . . . . . 296 11.2.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 11.3 Burst-trained equalizer with FQRD-RLS . . . . . . . . . . . . . . . . . . . . . . 300 11.3.1 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 11.3.2 Equivalent-output filtering . . . . . . . . . . . . . . . . . . . . . . . . . . 302 11.3.3 Equivalent-output filtering with explicit weight extraction 304 11.3.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 11.4 Active noise control and FQRD-RLS . . . . . . . . . . . . . . . . . . . . . . . . . 307 11.4.1 Filtered-x RLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 11.4.2 Modified filtered-x FQRD-RLS . . . . . . . . . . . . . . . . . . . . . . 309

xii

Contents

RE VI EW

11.4.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 11.5 Multichannel and lattice implementations . . . . . . . . . . . . . . . . . . . . . 313 11.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

12 Linear Constrained QRD-Based Algorithm . . . . . . . . . . . . . . . . . . . . . 317 Shiunn-Jang Chern 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 12.2 Optimal Linearly Constrained QRD-LS Filter . . . . . . . . . . . . . . . . . . 319 12.3 The Adaptive LC-IQRD-RLS Filtering Algorithm . . . . . . . . . . . . . . 321 12.4 The Adaptive GSC-IQRD-RLS Algorithm . . . . . . . . . . . . . . . . . . . . 324 12.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 12.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

UN DE R

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

RE VI EW

List of Contributors

Jos´e Antonio Apolin´ario Jr. (Editor) Department of Electrical Engineering (SE/3) Military Institute of Engineering (IME) Prac¸a General Tib´urcio 80, Rio de Janeiro, RJ, 22290-270 – Brazil e-mail: [email protected] Marcello L. R. de Campos Electrical Engineering Program, COPPE Federal University of Rio de Janeiro (UFRJ) P. O. Box 68504, Rio de Janeiro, RJ, 21941-972 – Brazil e-mail: [email protected]

UN DE R

Gilbert Strang Department of Mathematics Massachusetts Institute of Technology (MIT) 77 Massachusetts Avenue, Cambridge, MA 02139-4307 – USA e-mail: [email protected] Sergio L. Netto Electrical Engineering Program, COPPE Federal University of Rio de Janeiro (UFRJ) P. O. Box 68504, Rio de Janeiro, RJ, 21941-972 – Brazil e-mail: [email protected]

Maria D. Miranda Department of Telecommunications and Control University of S˜ao Paulo (USP) Avenida Prof. Luciano Gualberto 158, S˜ao Paulo, SP, 05508-900 – Brazil e-mail: [email protected]

xiii

xiv

List of Contributors

RE VI EW

Paulo S. R. Diniz Electrical Engineering Program, COPPE Federal University of Rio de Janeiro (UFRJ) P. O. Box 68504, Rio de Janeiro, RJ, 21941-972 – Brazil e-mail: [email protected]

Jenq-Tay Yuan Department of Electrical Engineering Fu Jen Catholic University 510 Chung Cheng Road, Hsinchuang, Taiwan 24205 – R.O.C. e-mail: [email protected] Ant´onio L. L. Ramos Department of Technology (ATEK) Buskerud University College (HIBU) P. O. Box 251, 3603 Kongsberg – Norway e-mail: [email protected]

Stefan Werner Department of Signal Processing and Acoustics, SMARAD CoE Helsinki University of Technology P.O. Box 3000 TKK, FIN-02015 – Finland e-mail: [email protected]

UN DE R

Athanasios A. Rontogiannis Institute for Space Applications and Remote Sensing National Observatory of Athens Metaxa and Vas. Pavlou Street, Athens 15236 – Greece e-mail: [email protected] Sergios Theodoridis Department of Informatics and Telecommunications University of Athens Panepistimiopolis, Ilissia, Athens 15784 – Greece e-mail: [email protected]

Phillip Regalia Department of Electrical Engineering and Computer Science Catholic University of America 620 Michigan Avenue NE, Washington, DC 20064 – USA e-mail: [email protected]

List of Contributors

xv

RE VI EW

Richard C. Le Borne Department of Mathematics Tennessee Technological University Box 5054, Cookeville, TN 38505 – USA e-mail: [email protected]

Marcio G. Siqueira Cisco Systems 170 West Tasman Drive, San Jose, CA 95134-1706 – USA e-mail: [email protected] Jun Ma School of Microelectronics Shanghai Jiaotong University 800 Dongchun Road, Shanghai 200240 – China e-mail: [email protected]

Keshab K. Parhi Department of Electrical and Computer Engineering University of Minnesota 200 Union Street SE, Minneapolis, MN 55455 – USA e-mail: [email protected]

UN DE R

Mohammed Mobien Department of Signal Processing and Acoustics, SMARAD CoE Helsinki University of Technology P.O. Box 3000 TKK, FIN-02015 – Finland e-mail: [email protected] Shiunn-Jang Chern Department of Electrical Engineering National Sun-Yat Sen University 70 Lienhai Road, Kaohsiung, Taiwan 80424 – R.O.C. e-mail: [email protected]

Chapter 11

RE VI EW

Weight Extraction of Fast QRD-RLS Algorithms Stefan Werner and Mohammed Mobien

UN DE R

Abstract The main limitation of FQRD-RLS algorithms is that they lack an explicit weight vector term. Furthermore, they do not directly provide the variables allowing for a straightforward computation of the weight vector as is the case with the conventional QRD-RLS algorithm, where a back-substitution procedure can be used to compute the coefficients. Therefore, the applications of the FQRD-RLS algorithms are limited to certain output-error based applications (e.g., noise cancelation), or to applications that can provide a decision-feedback estimate of the training signal (e.g., equalizers operating in decision-directed mode). This chapter presents some observations that allow us to apply the FQRD-RLS algorithms in applications that traditionally have required explicit knowledge of the transversal weights. Section 11.1 reviews the basic concepts of QRD-RLS and the particular FQRD-RLS algorithm that is used in the development of the new applications. Section 11.2 describes how to identify the implicit FQRD-RLS transversal weights. This allows us to use the FQRD-RLS algorithm in a system identification setup. Section 11.3 applies the FQRD-RLS algorithm to burst-trained systems, where the weight vector is updated during a training phase and then kept fixed and used for output filtering. Section 11.4 applies the FQRD-RLS algorithm for single-channel active noise control, where a copy of the adaptive filter is required for filtering a different input sequence than that of the adaptive filter. A discussion on multichannel and lattice extensions is provided in Section 11.5. Finally, conclusions are drawn in Section 11.6.

Stefan Werner Helsinki University of Technology, Espoo – Finland e-mail: [email protected] Mohammed Mobien Helsinki University of Technology, Espoo – Finland e-mail: [email protected]

291

292

Stefan Werner and Mohammed Mobien

11.1 FQRD-RLS Preliminaries

RE VI EW

To aid the presentation of the new FQRD-RLS applications, this section reviews the basic concepts of the QRD-RLS algorithm and one of the FQRD-RLS algorithms in Chapter 4, namely the FQR POS B algorithm.

11.1.1 QR decomposition algorithms

The RLS algorithm minimizes the following cost function k

ξ (k) = ∑ λ k−i [d(i) − xT(i)w(k)]2 = ke(k)k2 , i=0

(11.1)

where λ is the forgetting factor and e(k) ∈ R(k+1)×1 is the a posteriori error vector given as e(k) = d(k) − X(k)w(k), (11.2)

where d(k) ∈ R(k+1)×1 is the desired signal vector, X(k) ∈ R(k+1)×(N+1) is the input data matrix, and w(k) ∈ R(N+1)×1 . The QRD-RLS algorithm uses an orthogonal rotation matrix Q(k) ∈ R(k+1)×(k+1) to triangularize matrix X(k) as [1] 

 0 = Q(k)X(k), U(k)

(11.3)

where U(k) ∈ R(N+1)×(N+1) is the Cholesky factor of the deterministic autocorrelation matrix R(k) = XT (k)X(k). Pre-multiplying (11.2) with Q(k) gives      dq1 (k) 0 eq1 (k) = − w(k). Q(k)e(k) = U(k) eq2 (k) dq2 (k)

UN DE R



(11.4)

The cost function in (11.1) is minimized by choosing w(k) such that dq2 (k) − U(k)w(k) is zero, i.e., w(k) = U−1 (k)dq2 (k). (11.5) The QRD-RLS algorithm updates vector dq2 (k) and matrix U(k) as    d(k) eq1 (k) = Qθ (k) 1/2 , dq2 (k) λ dq2 (k − 1)

(11.6)

   xT (k) 0 . = Qθ (k) 1/2 λ U(k − 1) U(k)

(11.7)



and



11 Weight Extraction of Fast QRD-RLS Algorithms

293

where Qθ (k) ∈ R(N+2)×(N+2) is a sequence of Givens rotation matrices which annihilates the input vector x(k) in (11.7) and is partitioned as [2]  γ (k) gT (k) . Qθ (k) = f(k) E(k)

(11.8)

RE VI EW



The QRD-RLS algorithm is complete with the definition of the a priori error value e(k) = eq1 (k)/γ (k) where γ (k) is a scalar found in matrix Qθ (k), see (11.8).

11.1.2 FQR POS B algorithm

The idea of the FQRD-RLS algorithm is to replace the matrix update in (11.7) with a vector update. Using (11.5), we can express the a posteriori error ε (k) of the adaptive filter as T ε (k) = d(k) − x (k)U−1 (k) dq2 (k), (11.9) {z } | fT (k)

where

f(k) = U−T (k)x(k).

(11.10)

The FQR POS B algorithm, introduced in Chapter 4, updates vector f(k) by using forward and backward prediction equations and applying rotation matrices to triangularize the data matrix. The update is given by # # " " εb (k) f(k − 1) keb (k)k = Q′ (k) , (11.11) ε f (k) θf f(k) ke f (k)k

UN DE R

where ε f (k) and εb (k) are the a posteriori forward and backward prediction errors defined as

ε f (k) = x(k) − wTf (k)x(k − 1), εb (k) = x(k − N − 1) − wTb (k)x(k),

(11.12)

which are the first elements of the corresponding forward and backward prediction errors vectors, e f (k) and eb (k), given by (see Chapter 3)

294

Stefan Werner and Mohammed Mobien





RE VI EW

x(k)    λ 1/2 x(k − 1)  X(k − 1) X(k − 1)   w f (k) = d f (k) − w f (k), e f (k) =  − .. 01×(N+1) 01×(N+1)   . λ k/2 x(0)   x(k − N − 1) λ 1/2 x(k − N − 2)     .   − X(k)wb (k) = db (k) − X(k)wb (k). .. eb (k) =    (k−N−1)/2  λ x(0)  0(N+1)×1 (11.13) As seen in Chapter 4, vector w f (k) is not explicitly used for updating f(k). Instead, ε (k)

the term ke f (k)k in (11.11) is recursively computed by the FQRD-RLS algorithm. f The methods presented in the following will make explicit use of vector w f (k). Therefore, if we consider e f (k) in (11.13), then the vector w f (k) that minimizes ke f (k)k2 is given by    −1 X(k − 1) T w f (k) = XT (k − 1)X(k − 1) d f (k). 01×(N+1)

(11.14)

Let Q f (k − 1) denote the Givens rotation matrix defined by

T   Q(k − 1) 0k×1 Q(k − 1) 0k×1 = I, 01×k 1 01×k 1 | {z }



(11.15)

QTf (k−1)

UN DE R

where Q(k − 1) is the matrix that triangularizes X(k − 1), see (11.4). Applying Q(k − 1) and Q f (k − 1) to (11.14) yields −1  w f (k) = XT (k − 1)QT(k − 1)Q(k − 1)X(k − 1) T  X(k − 1) × QTf (k − 1)Q f (k − 1)d f (k) 01×(N+1) −1  = UT (k − 1)U(k − 1) UT (k − 1)d f q2 (k)

(11.16)

= U−1 (k − 1)d f q2 (k),

where d fq2 (k) corresponds to the last N + 1 elements (not taking into account the last element which corresponds to λ k/2 x(0)) of the rotated weighthed forward error vector defined as e fq = Q f (k − 1)e f (k).

11 Weight Extraction of Fast QRD-RLS Algorithms

295

RE VI EW

Equations (11.5), (11.11)–(11.12), and (11.16) are essential for the understanding of the weight extraction mechanism and output filtering methods explained in the following.

11.2 System Identification with FQRD-RLS

Unknown system

x(k)

n(k)

+

d(k)

+

Adaptive filter

y(k)

−

+

e(k)

Adaptation algorithm

Fig. 11.1 Schematic diagram of a system-identification application.

UN DE R

In many applications, it is necessary to identify an unknown system. Examples of such applications are: identification of the acoustic echo path in acoustic echo cancellation, channel identification in communications systems, and active noise control [1]. Figure 12.1 shows the basic structure of a system-identification application, where x(k), y(k), d(k), and e(k) are the input, output, desired output and error signals of the adaptive filter for time instant k. The adaptive filter and the unknown system share the same input signal, usually a wideband signal in the case of channel identification or a noisy voice signal in the case of acoustic echo cancellation. The adaptation algorithm compares the desired signal with the output of the adaptive filter in order to minimize the chosen objective function. The desired signal will, in addition to the output from the unknown system, contain some measurement noise n(k) which will affect the variance of the estimate of the unknown system. Let us now consider two possible approaches for identifying the unknown plant: one using an inverse QRD-RLS (IQRD-RLS) algorithm, with complexity O[N 2 ] per iteration, and another one that uses an FQRD-RLS algorithm with complexity

296

Stefan Werner and Mohammed Mobien

RE VI EW

O[N] per iteration. Obviously, the latter approach requires a mechanism in which the transversal weights embedded in the FQRD-RLS variables can be identified at any iteration of interest, e.g., after convergence. If the transversal weights are not required at every iteration, which might be the case in some applications, and the cost of extracting the FQRD-RLS weights is reasonably low, the overall computational complexity would be much lower with the second approach. The goal of this section is to develop a weight extraction (identification) mechanism that can be used in tandem with the FQRD-RLS algorithm at any particular iteration of interest. This would reduce the overall computational cost (and peakcomplexity) of the system identification.

11.2.1 Weight extraction in the FQRD-RLS algorithm

Note that the reduced computational cost of the FQRD-RLS algorithm is due to the fact that the matrix update equation (11.7) is replaced with the vector update equation (11.11). Thus, we are no longer able to separate U−1 (k) from x(k). As a consequence, if we excite the algorithm with a unit impulse, like in the serial weight flushing technique in [3, 4], (11.9) will not sequence out the correct coefficients. To approach the solution, we note from (11.5) that the ith coefficient of vector w(k) is given by wi (k) = dTq2 (k)ui (k),

(11.17)

UN DE R

where ui (k) denotes the ith column of matrix U−T (k). This means that when dq2 (k) is given, the elements of the weight vector w(k) can be computed if all the columns of matrix U−T (k) are known. In the following we show how all the column vectors ui (k) can be obtained in a serial manner given u0 (k). The main result is that the column vector ui (k) can be obtained from the column vector ui−1 (k) using two relations (denoted by →): ui−1 (k) → ui−1 (k − 1) → ui (k).

Let us first look at the relation ui−1 (k − 1) → ui (k). That is, assume that we are given the (i − 1)th column of U−T (k − 1) (please note the index value k − 1). We can then compute the ith column of U−T (k) using (11.11) as stated in Lemma 1.

11 Weight Extraction of Fast QRD-RLS Algorithms

297

where ∗ is a “don’t-care” and

RE VI EW

Lemma 1: Let ui (k) ∈ R(N+1)×1 denote the ith column of the upper triangular matrix U−T (k) ∈ R(N+1)×(N+1) . Given Q′θ f (k) ∈ R(N+2)×(N+2) , d f q2 (k) ∈ R(N+1)×1 , and ke f (k)k from the FQRD-RLS algorithm, then ui (k) can be obtained from ui−1 (k − 1) using the following relation # "   ui−1 (k − 1) ∗ ′ = Qθ f (k) −w f ,i−1 (k) , i = 0, . . . , N, (11.18) ui (k) ke f (k)k

w f ,i−1 (k) =



−1 for i = 0, uTi−1 (k − 1)d f q2 (k) otherwise.

(11.19)

Equation (11.18) is initialized with u−1 (k − 1) = 0(N+1)×1 .

The definitions in (11.10) and (11.12) allow us to rewrite (11.11) as # " T # " −wb (k) 0 U−T (k − 1) 1 (N+2) ′ keb (k)k keb (k)k x x(N+2) (k), (11.20) (k) = Q θ f (k) −wTf (k) 1 −T U (k) 0 ke f (k)k ke f (k)k

UN DE R

where x(N+2) (k) = [x(k) x(k − 1) · · · x(k − N − 1)]T. Equation (11.20) is illustrated in Figure 12.5, where we see that (11.18)–(11.19) are just the column description of the matrices multiplying the extended input vector x(N+2) (k). To account for the first column of (11.20) we initialize with u−1 (k − 1) = 0N×1 and w f ,−1 (k) = −1, which can be clearly seen from Figure 12.5. Consequently, the first coefficient can be computed directly following the initialization as (11.21) w0 (k) = dTq2 (k)u0 (k). To proceed with the next coefficient w1 (k) = dTq2 (k)u1 (k), we need vector u0 (k − 1) to compute u1 (k) using (11.18). In general, having computed wi (k), i.e., ui (k) is known, we need a reverse mechanism, ui (k) → ui (k − 1), in tandem with (11.18)– (11.19) to allow for the computation of the next weight wi+1 (k). How to compute ui (k − 1) from ui (k) is summarized by Lemma 2. Equations (11.22)–(11.23) are obtained directly from the update equation for U−T (k − 1), see Chapter 3.

298

Stefan Werner and Mohammed Mobien

where

RE VI EW

Lemma 2: Let ui (k) ∈ R(N+1)×1 denote the ith column of the upper triangular matrix U−T (k) ∈ R(N+1)×(N+1) . Given Qθ (k) ∈ R(N+2)×(N+2) , f(k) ∈ R(N+1)×1 and γ (k) from the FQRD-RLS algorithm, then ui (k − 1) can be obtained from ui (k) using the relation below     0 zi (k) T = Qθ (k) , i = 0, . . . , N − 1, (11.22) λ −1/2 ui (k − 1) ui (k) zi (k) = −fT (k)ui (k)/γ (k).

(i)th column

0

Q′θ f (k)

0

U−T (k − 1) T

−T

−d f q2 (k)U (k−1) 1 ke f (k)k ke f (k)k

−dTbq2 (k)ui (k) keb (k)k

ui (k)

ui−1 (k − 1)

Corresponding vectors

x(N+2) (k)

U−T (k)

1 keb (k)k

(i)th column

x(N+2) (k)

−dTbq2 (k)U−T (k) keb (k)k

(11.23)

UN DE R

−dTf q2 (k)ui−1 (k−1) ke f (k)k

Fig. 11.2 Illustration of (11.18) on how to obtain the ith column of U−T (k) (denoted ui (k)) from the (i − 1)th column of U−T (k − 1) (denoted ui−1 (k − 1)).

The update is given by [5]    T  z (k) 0T = Q . (k) θ λ −1/2 U−T (k − 1) U−T (k)

(11.24)

Equation (11.22) is obtained by pre-multiplying both sides of (11.24) with QTθ (k) and considering each column separately, zi (k) being the ith element of vector z(k). If we now employ the standard partition of Qθ (k) in (11.8), we can directly verify that the first element of (11.22) is equal to 0 = γ (k)zi (k) + fT (k)ui (k),

(11.25)

11 Weight Extraction of Fast QRD-RLS Algorithms

299

Table 11.1 FQRD-RLS weight extraction algorithm. Weight Extraction

RE VI EW

Initialization: xi = 0, ∀i ∈ [1, N] x0 = −1 u−1 (k − 1) = 0(N+1)×1

Available from the FQRD-RLS algorithm:

Q′θ f (k), d f q2 (k), f(k), ke f (k)k, Qθ (k), γ (k), and dq2 (k) for each i = 0 to N {

Compute ui (k) from ui−1 (k − 1):

w f ,i−1(k) = xi − uTi−1 " (k − 1)d f q2#(k)   ui−1 (k − 1) ∗ = Q′θ f (k) −w f ,i−1 (k) ui (k) ke (k)k f

Compute ui (k − 1) from ui (k):

zi (k) = −fT (k)ui(k)/γ (k)    0 T (k) z j (k) = Q θ ui (k) λ −1/2 ui (k − 1) Compute wi (k):

wi (k) = dTq2 (k)ui (k) }

Table 11.2 Computational complexity of weight extraction (WE) [8]. ALGORITHM

MULT

DIV

SQRT

UN DE R

FQRD-RLS 19N + 23 4N + 5 2N + 3 WE (per weight i, 0 ≤ i ≤ N) 11N + 14 − 11i 0 0 WE (total) 5.5N 2 + 19.5N + 7 0 0 IQRD-RLS 3N 2 + 8N + 4 2N + 2 N + 1

which gives the relation in (11.23). In summary, (11.18) and (11.22) allow for a serial weight extraction of the weights w(k) embedded in the FQRD-RLS variables. The pseudo-code for the weight extraction mechanism is given in Table 11.1. The number of operations required to completely extract all the coefficients is given in Table 11.2. For comparison, the computational costs of the FQRD-RLS and the IQRD-RLS algorithms are also given.

300

Stefan Werner and Mohammed Mobien FQRD-RLS IQRD-RLS

−5

−305

∆ w2i (k) (dB)

min MSE −310

−315

−15

RE VI EW

MSE (dB)

−10

−320

−20

−325

−25

−330

−30 200

400

600

800

1000 1200 1400 1600 1800

Iteration, k

−335 0

2

4

6

8

Coefficient, i

10

Fig. 11.3 Learning curves of the FQRD-RLS and the IQRD-RLS algorithms (left figure), squared difference between coefficient weights of the IQRD-RLS and the FQRD-RLS algorithms.

11.2.2 Example

The FQRD-RLS and IQRD-RLS algorithms, both using λ = 0.95, were used to identify a system of order N = 10. The input signal x(k) was a noise sequence, colored by filtering a zero-mean white Gaussian noise sequence nx (k) through the fourth-order IIR filter x(k) = nx (k) + x(k − 1) + 1.2x(k − 2) + 0.95x(k − 3), and the SNR was set to 30dB. After 1900 iterations, the internal variables that are required for computing w(k) were saved. The transversal weight vector of the IQRD-RLS algorithm and the weight vector extracted from the FQRD-RLS algorithm are compared by taking the squared difference of each coefficient, i.e.,

∆ w2i (k) = |wFQRD,i (k) − wIQRD,i (k)|2 .

(11.26)

UN DE R

Figure 11.3 shows the learning curves and the weight difference after 1900 iterations. We see that the two algorithms have identical learning curves, and that the transversal weight vector extracted from the FQRD-RLS algorithm is identical to that of the IQRD-RLS algorithm up to the simulation precision.

11.3 Burst-trained equalizer with FQRD-RLS In wireless communications systems, the main factors limiting the system capacity are various kinds of interference such as intersymbol interference (ISI) due to multipath propagation in frequency selective fading channels, co-channel (or multiple access) interference, and adjacent channel interference. ISI is the main impairment in single user communications and can be corrected through the use of an adaptive equalizer [6]. Figure 11.4 shows the structure of an adaptive equalizer, where

PSfrag 11 Weight Extraction of Fast QRD-RLS Algorithms

301

i(k) + n(k) Channel C(z)

+

x(k)

Adaptive filter w(k − 1)

y(k)

RE VI EW

u(k)

Adaptation algorithm

z−D

d(k)

+

+

e(k) −

Fig. 11.4 Schematic diagram of an adaptive equalizer.

u(k) is the user signal of interest and i(k) is co-channel interference. The adaptive filter will try to suppress the channel-induced ISI, and in certain applications also the co-channel interference. The desired signal d(k) is now a delayed replica of the transmitted signal, where the value of the delay D is chosen to compensate for the delay introduced by the channel. In this section we will examine the special case of burst-trained equalizers, where the equalizer coefficients are periodically updated using known training symbols and then used for fixed filtering of a useful data sequence.

11.3.1 Problem description

UN DE R

The problem under consideration is illustrated in Figure 11.5. During time instants k ≤ k f , the equalizer operates in training mode and its coefficients are updated using the input and desired signal pair {x(k), d(k)}. At time instant k = k f , the adaptive process is stopped and the equalizer switches to data mode where the coefficient vector obtained during the training mode is kept fixed. That is, the output of the filter is given by  T w (k − 1)x(k) k ≤ k f (11.27) y(k) = wT (k f )x(k) k > k f

where w(k f ) is the coefficient vector of the adaptive filter “frozen” at time instant k = kf . If an FQRD-RLS algorithm is employed during training, one alternative for carrying out the filtering during the data mode, k > k f , is to first extract the filter coefficients according to Section 11.2 and, thereafter, perform the filtering of x(k) with

302

Stefan Werner and Mohammed Mobien

k = kf

d(k)

x(k)

RE VI EW

e(k)

y(k)

w(k − 1)

Fig. 11.5 Operation of a burst-trained equalizer. The adaptive filter coefficients are updated during training mode (k ≤ k f ), and kept fixed and used for output filtering in data mode (k > k f ). Note that there is no weight update after k > k f .

a simple transversal structure. To avoid the increased peak complexity O[N 2 ] of this solution (at time k f ), we seek here alternative methods with reduced peak complexity O[N] that can reproduce the output signal in (11.27) from the variables of the FQRD-RLS algorithm available at instant k = k f [7].

11.3.2 Equivalent-output filtering

After the training of the FQRD-RLS algorithm is stopped (k = k f ), filtering of the useful signal should be carried out according to (compare with (11.27)) y(k) = dTq2 (k f )U−T (k f ) x(k), k > k f , | {z }

(11.28)

wT (k f )

UN DE R

where dq2 (k f ) and U−T (k f ) are parameters of the FQRD-RLS algorithm at time instant k = k f , respectively. Vector dq2 (k f ) in (11.28) is explicitly available in the FQRD-RLS algorithm. However, knowledge of U−T (k f ) is only provided through the variable f(k f ) = U−T (k f )x(k f ). Thus, starting with f(k f ) as an initial value, we need to find a way to obtain vector U−T (k f )x(k) from vector U−T (k f )x(k − 1), i.e., we need to incorporate the new sample x(k) without affecting U−T (k f ) in the process. This problem is somewhat similar to the weight extraction problem that was treated in the previous section. The solution exploits the following two steps: U−T (k f )x(k − 1) → U−T (k f − 1)x(k − 1) → U−T (k f )x(k).

The first step, summarized by Lemma 3, is obtained by pre-multiplying (11.24) with QTθ (k f ) and post-multiplying with the vector [x(k) xT (k − 1)]T . The variable z(k) in (11.31) is obtained in a similar manner as (11.24). That is, by employing the

11 Weight Extraction of Fast QRD-RLS Algorithms

303

RE VI EW

partition of Qθ (k f ) in (11.8), the equation describing the first element of (11.30) is given by (11.29) 0 = γ (k f )z(k) + fT (k f )U−T (k f )x(k − 1). Lemma 3: Let U−T (k f ) ∈ R(N+1)×(N+1) denote the upper triangular inverse transpose Cholesky matrix corresponding to time instant k f , and x(k − 1) ∈ R(N+1)×1 be the input vector at any instant k > k f . Given Qθ (k f ) ∈ R(N+2)×(N+2) , f(k f ) ∈ R(N+1)×1 , and γ (k f ) from the FQRD-RLS algorithm, then U−T (k f − 1)x(k − 1) is obtained from U−T (k f )x(k − 1) using the relation below 

0



λ −1/2U−T (k f − 1)x(k − 1)

where

= QTθ (k)



 z(k) , U−T (k f )x(k − 1)

z(k) = −fT (k f )U−T (k f )x(k − 1)/γ (k f ).

(11.30)

(11.31)

The second step, summarized by Lemma 4, is obtained from (11.20) and freezing the FQRD-RLS variables at k = k f , i.e., "

−wTb (k f ) 1 keb (k f )k keb (k f )k U−T (k f ) 0

#

x(N+2) (k) = Q′ θ f (k f )

"

0

1 ke f (k f )k

# U−T (k f − 1) −wTf (k f ) ke f (k f )k

x(N+2) (k).

(11.32) We see that the extended input vector x(N+2) (k) multiplies both sides of (11.32). Therefore, the time instant for x(N+2) (k) can be chosen arbitrarily.

UN DE R

Lemma 4: Let U−T (k f − 1) ∈ R(N+1)×(N+1) denote the upper triangular inverse transpose Cholesky matrix corresponding to time instant k f − 1, and x(k − 1) ∈ R(N+1)×1 be the input vector at any instant k > k f . Given Q′θ f (k f ) ∈ R(N+2)×(N+2) , d f q2 (k f ) ∈ R(N+1)×1 , and ke f (k f )kfrom the FQRD-RLS algorithm and input sample x(k), then U−T (k f )x(k) is obtained from U−T (k f − 1)x(k − 1) as 

# "  U−T (k f − 1)x(k − 1) ∗ = Q′θ f (k f ) x(k)−dTf q2 (k f )U−T (k f −1)x(k−1) , U−T (k f )x(k) ke (k )k f

(11.33)

f

where ∗ is a “don’t-care” variable. In summary, (11.30) and (11.33) allow us to reproduce the equivalent-output signal corresponding to (11.28) without explicit knowledge of the weights embedded in the FQRD-RLS algorithm. The procedure is illustrated in Figure 12.7. The pseudocode of for the equivalent-output filtering algorithm is provided in Table 11.3.

304

Stefan Werner and Mohammed Mobien

time k:

Eq. (11.33) U−T (k f − 1)x(k − 1)

U−T (k f )x(k)

RE VI EW

y(k) = dTq2 (k f )U−T (k f )x(k)

Eq. (11.30) k+1 :

Eq. (11.33) U−T (k f − 1)x(k)

U−T (k f )x(k + 1)

y(k + 1) = dTq2 (k f )U−T (k f )x(k + 1)

Eq. (11.30)

.. .

Fig. 11.6 Fixed filtering without explicit knowledge of weight vector embedded in the FQRD-RLS algorithm.

11.3.3 Equivalent-output filtering with explicit weight extraction

The approach presented here is based on the observation that for k > k f we can divide the input vector x(k) into two non-overlapping vectors c(k) ∈ C(N+1)×1 and v(k) ∈ C(N+1)×1 x(k) = c(k) + v(k), k > k f , (11.34) with initial values

UN DE R

c(k f ) = 0 v(k f ) = x(k f ),

(11.35)

where c(k) contains the input-samples from k > k f and v(k) holds those remaining from k ≤ k f . In other words, for each time instant k the new input-sample x(k) is shifted into c(k) and a zero is shifted into v(k). The output y(k) for k > k f can now be written as y(k) = yc (k) + yv (k) = wT (k f )c(k) + wT(k f )v(k).

(11.36)

Note that with the initialization in (11.35), v(k) = 0 and y(k) = yc (k) for k > k f + N.

The above formulation allows us to make use of our previous results and divide the problem into two parts that can be carried out in parallel: one distributed weight

11 Weight Extraction of Fast QRD-RLS Algorithms

305

Table 11.3 Reproducing output y(k) = wT (k f )x(k) (k > k f ) from the FQRD-RLS variables. Equivalent-Output Filtering

RE VI EW

Initialization: r(k) = f(k f ) ≡ U−T (k f )x(k f )

Available from the FQRD-RLS algorithm:

Q′θ f (k f ), d f q2 (k f ), f(k f ), ke f (k f )k, Qθ (k f ), γ (k f ), and dq2 (k f ) for each k > k f {

Compute U−T (k f − 1)x(k − 1) from U−T (k f )x(k − 1):    T  0 −f (k f )r(k − 1)/γ (k f ) = QTθ (k f ) −1/2 r(k − 1) λ r˜ (k) Compute U−T (k f )x(k) from U−T (k f − 1)x(k − 1): " #   r˜ (k) ∗ ′ T = Qθ f (k f ) x(k)−d f q2 (k f )˜r(k) r(k) ke (k )k f

Compute y(k)

= wT (k

f

f )x(k):

y(k) = dTq2 (k f )r(k) }

UN DE R

extraction that is used with c(k) to produce yc (k), and; one equivalent-output part reproducing yv (k). Reproducing yv (k) is straightforward. We need only to apply the results in previous subsection using vector v(k) in place of x(k). Obtaining yc (k) and w(k f ) is based on the observation that yc (k) during the first N + 1 iterations, following the training phase (k > k f ), is given by k−(k f +1)

yc (k) =

∑

wi (k f )c(k − i), k f < k ≤ k f + N + 1,

(11.37)

i=0

where c(k) = x(k) ∀k > k f and c(k) = 0 ∀k ≤ k f . We see that (11.37) allows us to extract the weights in a distributed manner, i.e., one weight per each new incoming sample. Such ”on-the-fly” extraction provides us with all the weights after N + 1 iterations, and still produces the correct output yc (k) before all the weights are acquired (according to (11.37)). Invoking Lemmas 1 and 2 using a unit pulse in parallel with (11.37) will sequence out the weights wi (k f ) at the time instant they show up in (11.37). After the initial N + 1 iterations, the output y(k) is simply given by wT (k f )x(k) = T w (k f )c(k). In other words, it is not necessary to make all the weights available

306

Stefan Werner and Mohammed Mobien 0 FQRD-RLS IQRD-RLS Min. MSE

training mode

−10

RE VI EW

MSE (dB)

−5

data mode

−15

−20

−25

−30 0

100

200

300

400

500

600

700

800

900

Iteration, k

Fig. 11.7 Learning curves of the FQRD-RLS and the IQRD-RLS algorithms.

before starting filtering in data mode (peak complexity O[N 2 ] [8]). This distributed weight flushing procedure ensures a peak complexity of O[N].

11.3.4 Example

The channel equalization example is taken from [9], where the channel is given by C(z) = 0.5 + 1.2z−1 + 1.5z−2 − z−3 .

UN DE R

The SNR is set to 30 dB and the order of the equalizer is N = 49. During the first 150 iterations (i.e., k f = 150), the equalizer coefficients are updated by the FQRD-RLS algorithm. The training symbols d(k) were randomly generated BPSK symbols. Following the initial training sequence, an unknown symbol sequence consisting of 750 4-PAM symbols was transmitted over the channel and the equalizer output was reproduced using the approach in Section 11.3.3. For comparison purposes, an IQRD-RLS algorithm was also implemented. Note that the IQRD-RLS has a complexity of O[N 2 ] during coefficient adaptation. Figure 11.7 shows the MSE curves for the FQRD-RLS and the IQRD-RLS approaches. The results were obtained by averaging and smoothing 100 realizations of the experiment. It can be seen that both algorithms converge to the same solution.

11 Weight Extraction of Fast QRD-RLS Algorithms x(k)

307 d(k)

Primary path

Adaptive filter w(k − 1)

RE VI EW

P(z)

y(k)

Secondary path

y f (k)

+

−

e(k)

+

S(z)

Fig. 11.8 Schematic diagram of an active noise control (ANC) system.

11.4 Active noise control and FQRD-RLS

UN DE R

In active noise control, a disturbing (primary) noise is canceled by generating an “anti-noise” signal with identical amplitude and opposite phase [10]. Figure 11.8 shows a single-channel system consisting of one reference sensor (microphone) measuring the noise signal x(k), one actuator (loudspeaker) signal y(k), and one error sensor (microphone) measuring the residual signal e(k). In the figure, P(z) is the primary-path response, i.e., the (acoustic) response from the noise sensor to the error sensor, and S(z) is the secondary-path response that models the acoustic path between the actuator and error microphone as well as other imperfections like D/A and A/D converters, reconstruction filters, power amplifier effects [10]. Assuming that S(z) is known, the task of the adaptive filter is to identify the unknown primary path P(z).

11.4.1 Filtered-x RLS

The error signal e(k) observed by the error microphone is given by e(k) = d(k) − y f (k) = d(k) − s(k) ∗ [xT(k)w(k − 1)], {z } |

(11.38)

y(k)

where ∗ denotes convolution, s(k) is the impulse response of the secondary path S(z), and y(k) = ∑Ni=0 x(k − i)wi (k − 1) = x(k) ∗ w, wi (k − 1) being the i-th element of w(k − 1) and w representing the impulse response of the adaptive filter at k − 1. Knowing that s(k) ∗ [x(k) ∗ w] = [s(k) ∗ x(k)] ∗ w, we define the filtered input signal vector x f (k) = [x f (k) x f (k − 1) · · · x f (k − N)]T , (11.39)

308

Stefan Werner and Mohammed Mobien

whose elements are given as delayed versions of x f (k) = s(k) ∗ x(k). We can now formulate the WLS objective function as k

k

i=0

(11.40)

RE VI EW

Jw = ∑ λ k−i ε 2 (i) = ∑ λ k−i [d(i) − x f T (i)w(k)]2 . i=0

Differentiating the objective function Jw with respect to w(k) and solving for the minimum results in w(k) = R−1 f (k)p f (k), where R f (k) and p f (k) are defined by k

R f (k) = ∑ λ k−i x f (i)xTf (i), and i=0 k

p f (i) = ∑ λ

(11.41)

k−i

x f (i)d(i).

i=0

The filtered-x RLS (FX-RLS) algorithm is then obtained in a similar manner as the conventional RLS algorithm in Chapter 2 by substituting R f (k) and p f (k) in (??) with their recursive formulations R f (k) = λ R f (k) + x f (k)x f T (k), and

(11.42)

p f (k) = λ p f (k) + d(k)x f (k),

(11.43)

giving the following updating expression

w(k) = w(k − 1) + e(k)R f −1 (k)x f (k).

(11.44)

UN DE R

−1 The inverse R−1 f (k) can be obtained recursively in terms of R f (k − 1) using the 1 matrix inversion lemma [1] thus avoiding direct inversion of R f (k) at each time instant k. Note that x f (k) depends on the impulse response s(k) of the secondary path S(z). ˆ of S(z) can be obtained offline during an For most ANC systems, an estimate S(z) initial training phase [10]. Then the filtered input signal vector x f (k) used with the FX-RLS algorithm is given by

x f (k) = s(k) ˆ ∗ x(k),

ˆ where s(k) ˆ is the impulse response of S(z).

(11.45)

11 Weight Extraction of Fast QRD-RLS Algorithms

x(k)

309

d(k)

Primary path

Adaptive filter copy

RE VI EW

P(z)

y(k)

Secondary path

y f (k)

+

−

e(k)

+

S(z)

yˆ f (k)

ˆ S(z)

+

ˆ d(k)

x f (k)

ˆ S(z)

FQRD-RLS algorithm

Fig. 11.9 FQRD-RLS in an active noise control (ANC) system.

11.4.2 Modified filtered-x FQRD-RLS

UN DE R

The main problems with the FX-RLS algorithm are potential divergence behavior in finite-precision environment and high computational complexity, which is of order N 2 . As an alternative, we could think of a more robust solution with reduced complexity that employs the FQRD-RLS algorithm. However, the FQRD-RLS algorithm (and also standard QRD-RLS algorithms) requires explicit knowledge of d(k) to minimize the objective function in (11.40). This should be compared with the FX-RLS implementation in (11.44) that directly employs the error signal e(k) measured by the error microphone. On the other hand, we see from Figure 11.8 that ˆ if we pass the actuator signal y(k) through the estimated secondary-path filter S(z), ˆ ˆ ∗ y(k), an estimate d(k) can be obtained as i.e., we obtain yˆ f (k) = s(k) ˆ = e(k) + yˆ f (k) = e(k) + s(k) ˆ ∗ y(k). d(k)

(11.46)

This leads to the realization in Figure 11.9. This structure, referred to as the modified filtered-x structure and has been used with conventional RLS and LMS

1

[A + BCD]−1 = A−1 − A−1 B[DA−1 B + C−1 ]−1 DA−1 .

310

Stefan Werner and Mohammed Mobien

RE VI EW

algorithms as well as with the IQRD-RLS algorithm [11] to improve convergence speed and robustness. We see from Figure 11.9 that the coefficient vector embedded in the FQRDRLS variables is needed for reproducing the output signal y(k) = wT (k − 1)x(k). We know from Section 11.2 that w(k − 1) can be made explicitly available at every iteration. However, as can seen from Table 11.2 such an approach will lead to a solution of O[N 2 ] complexity per iteration. In other words, there is no obvious gain of using an FQRD-RLS algorithm in place of an IQRD-RLS algorithm. One solution to this complexity problem is to extract and copy the weights at a reduced rate, say once every K samples. This type of intermediate solution was considered in [11], where a QRD least-squares lattice (QRD-LSL) algorithm was employed in the lower branch of Figure 11.9. Obviously, such an approach will no longer yield identical results to an IQRD-RLS algorithm, and the convergence behavior will certainly be different. Our goal here is to reproduce, at each iteration, the exact output y(k) associated with the weight vector w(k − 1) embedded in the FQRD-RLS algorithm. The total computational complexity for calculating y(k) and updating w(k − 1) will be O[N] per iteration. The resulting structure will yield exactly the same result as the modified filtered-x IQRD-RLS in algorithm [11], while reducing the computational complexity by an order of magnitude.

UN DE R

The solution is very similar to the problem dealt with in Section 11.3 where the weight vector in the FQRD-RLS algorithm was used for fixed filtering. The main difference here is that the weight vector in the lower branch is continuously updated by the FQRD-RLS algorithm. The output y(k) can be expressed as y(k) = wT (k − 1)x(k) = dTq2 (k − 1)U−T(k − 1)x(k),

(11.47)

where, dq2 (k − 1) and U−T (k − 1) are parameters of the FQRD-RLS algorithm running in the lower branch of Fig 11.9. The rotated desired signal vector dq2 (k) is directly available in the FQRD-RLS algorithm. However, the Cholesky factor matrix U−1 (k − 1) is hidden in vector f(k − 1) = UT (k − 1)x f (k − 1). On the other hand, we know from (11.20) that matrix Q′θ f (k − 1) provides the relation UT (k − 2) → UT (k − 1). Since vectors x f (k) and x(k) are both initially set to zero, we can use (11.20) for calculating y(k) for k > 0. The required computations are summarized by Lemma 5.

11 Weight Extraction of Fast QRD-RLS Algorithms

311

Table 11.4 Modified filtered-x FQRD-RLS algorithm. Output filtering in the upper branch of Figure 11.9

RE VI EW

Initialization: r(0) = UT (−1)x(0) = 0(N+1)×1

Available from FQRD-RLS algorithm:

Q′θ f (k − 1), d f q2 (k − 1), and ke f (k − 1)k for each k {

Compute U−T (k − 1)x(k) from U−T (k − 2)x(k − 1): " #   r(k − 1) ∗ ′ T = Qθ f (k − 1) x(k)−d f q2 (k)r(k−1) r(k) ke (k−1)k f

Compute y(k) = wT (k − 1)x(k): y(k) = dTq2 (k)r(k) }

Lemma 5: Let U−T (k −1) ∈ R(N+1)×(N+1) denote the upper triangular inverse transpose Cholesky matrix corresponding to the filtered autocorrelation matrix R f (k − 1) defined in (11.41), and x(k − 1) ∈ R(N+1)×1 be the input vector at any instant k > 0. Given Q′θ f (k − 1) ∈ R(N+2)×(N+2) , d f q2 (k − 1) ∈ R(N+1)×1 , and ke f (k − 1)k from the FQRD-RLS algorithm operating in the lower branch of Figure 11.9, then U−T (k − 1)x(k) in (11.47) obtained from U−T (k − 2)x(k − 1) as # "  U−T (k − 2)x(k − 1) ∗ = Q′θ f (k − 1) x(k)−dTf q2 (k−1)U−T (k−2)x(k−1) , U−T (k − 1)x(k) ke (k)k

UN DE R



(11.48)

f

where ∗ is a “don’t-care” variable. The algorithm for reproducing the output y(k) in the upper branch of Figure 11.9 is summarized in Table 11.4.

11.4.3 Example

To illustrate the modified filtered-x FQRD-RLS algorithm, we consider an ANC setup where the primary-path and secondary-path filters are given as follows.

312

Stefan Werner and Mohammed Mobien 0 FQRD-RLS FQRD-RLS (WE) K = 50 FQRD-RLS (WE) K = 20 IQRD-RLS

−20

−30

−40

−50

−60 0

100

RE VI EW

MSE (dB)

−10

200

300

400

500

Iteration, k

Fig. 11.10 Learning curves of the FQRD-RLS and IQRD-RLS algorithms implemented in the modified filtered-x structure. For comparison purposes, FQRD-RLS weight extraction (WE) solutions are shown where the coefficients are extracted at a reduced rate, every K = 20 or K = 50 iterations, and copied to the upper branch.

P(z) = 0.4828z−3 − 0.8690z−4 + 0.0966z−5 + 0.0483z−6 S(z) = 0.8909z−2 + 0.4454z−3 + 0.0891z−4

(11.49)

UN DE R

The order of the adaptive filter is N = 19, and the input signal x(k) was a colored noise sequence, colored by filtering a zero-mean white Gaussian noise sequence nx (k) through the third-order IIR filter x(k) = −1.2x(k − 1) − 0.95x(k − 2) + nx(k). The primary signal d(k) was further disturbed by an additive noise sequence n(k) whose variance was set such that the SNR at the error microphone was 28 dB. For comparison purposes, we implemented the IQRD-RLS algorithm [11] and an FQRD-RLS solution where weight extraction (WE) is performed in the lower branch at a reduced rate, once every K = 20 or K = 50 iterations, and the extracted coefficients are then copied to the upper branch and kept fixed until the next WE takes place. Figure 11.10 shows the MSE curves for the FQRD-RLS and the IQRDRLS implementations. The results were obtained by averaging and smoothing 50 realizations of the experiment. The FQRD-RLS and the IQRD-RLS algorithms have identical converge behavior (up to machine precision). As expected, the convergence behavior changes when considering a solution that extracts the weights at a reduced rate. In case of a non-stationary primary path (which may be not uncommon in practical applications), this difference would be more relevant.

11 Weight Extraction of Fast QRD-RLS Algorithms

313

11.5 Multichannel and lattice implementations

UN DE R

RE VI EW

The results presented in this chapter were based on single-channel adaptive FIR structures. All the results presented in this chapter can be extended to multichannel FQRD-RLS algorithms, e.g., [9], [12]–[15]. Thus, enabling multichannel applications such as broadband beamforming [16], Volterra system identification [9], Volterra predistorters [17], burst-trained decision-feedback (DFE) and Volterra equalizers [1], multichannel active noise control [18], to name but a few. The result for weight extraction can also be extended to fast QR-decomposition algorithms belonging to the family of least-squares lattice based algorithms, e.g., the QRD-LSL algorithms proposed in [19, 20]. The problem of parameter identification in fast RLS algorithms has been previously addressed using the duality between the FQRD-RLS algorithms and the normalized Lattice structure [19]. In other words, the weight extraction for the fast QR-decomposition can be solved by finding the reflection coefficients and the backward prediction weight counterparts using duality [19]. It was shown in [4] and indicated in [19] that, by using the least-squares lattice version of the Levinson-Durbin recursion, the transversal weights can be obtained from the variables of the QRD-LSL algorithm at a cost of O[N 3 ] operations. In practice, the high cost of the weight identification using an exact Levinson-Durbin algorithm can be avoided by assuming algorithm convergence and infinite memory support, i.e., λ = 1 [21].The computational complexity is then reduced at the cost of accuracy of the solution. A modification of Lemma 1 in Section 11.2 was introduced in [22] to allow for the transversal weight identification at any chosen iteration with a computational complexity of O[N 2 ] without compromising accuracy. The main observation, was that the order-update required by the exact Levinson-Durbin algorithm can be done in one step by exploiting the known QRD-LSL variables, reducing the number of required operations by an order of magnitude.

11.6 Conclusion

This chapter showed how to reuse the internal variables of the fast QRD-RLS (FQRDRLS) algorithm to enable new applications which are different to the standard outputerror type applications. We first considered the problem of system identification and showed how to extract the weights in a serial manner. Thereafter, the weight extraction results were extended to the case of burst trained equalizers, where the equalizer is periodically re-trained using pilots and then used for fixed filtering of useful data. Finally, we considered the problem of active noise control, where a modified filtered-x FQRD-RLS structure was introduced. Simulation results were compared with those using a design based on the inverse QRD-RLS algorithm. It was verified

314

Stefan Werner and Mohammed Mobien

that, with the help of the WE techniques detailed in this chapter, identical results are obtained using the FQRD-RLS methods at a much lower computational cost.

References

RE VI EW

Acknowledgements This work was partially funded by the Academy of Finland, Smart and Novel Radios (SMARAD) Center of Excellence.

UN DE R

1. P. S. R. Diniz, Adaptive Filtering: Algorithms and Practical Implementation. 3rd edition Springer, New York, NY, USA (2008) 2. J. A. Apolin´ario Jr. and P. S. R. Diniz, A new fast QR algorithm based on a priori errors. IEEE Signal Processing Letters, vol. 4, no. 11, pp. 307–309 (November 1997) 3. C. R. Ward, A. J. Robson, P. J. Hargrave, and J. G. McWhirter. Application of a systolic array to adaptive beamforming. IEE Proceedings, Part F - Communications, Radar and Signal Processing, vol. 131, no. 6, pp. 638–645 (October 1984) 4. S. Haykin, Adaptive Filter Theory. 3rd edition, Prentice-Hall, Englewood Cliffs USA (1996) 5. S.T.Alexander and A.L. Ghirnikar, A method for recursive least squares filtering based upon an inverse QR decomposition. IEEE Transactions on Signal Processing, vol. 41, no. 1, pp. 20– 30 (January 1993) 6. S. U. Qureshi, Adaptive equalization. Proceedings of the IEEE, vol. 73, no. 9, pp. 1349–1387 (September 1985) 7. M. Shoaib, S. Werner, J. A. Apolin´ario Jr., and T. I. Laakso, Equivalent output-filtering using fast QRD-RLS algorithm for burst-type training applications. IEEE International Symposium on Circuits and Systems, ISCAS’2006, Kos, Greece (May 2006) 8. M. Shoaib, S. Werner, J. A. Apolin´ario Jr., and T. I. Laakso, Solution to the weight extraction problem in FQRD-RLS algorithms. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’2006, Toulouse, France, pp. 572–575 (May 2006) 9. M.A. Syed and V.J. Mathews, QR-decomposition based algorithms for adaptive Volterra filtering. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, vol. 40, no. 6, pp. 372–382 (June 1993) 10. S. M. Kuo and D. R. Morgan, Active noise control: a tutorial review. Proceedings of the IEEE, vol. 87, no. 6, pp. 943–973 (June 1999) 11. M. Bouchard, Numerically stable fast convergence least-squares algorithms for multichannel active sound cancellation systems and sound deconvolution systems. Signal Processing, vol. 82, no. 5, pp. 721–736 (May 2002) 12. A. A. Rontogiannis and S. Theodoridis, Multichannel fast QRD-LS adaptive filtering: new technique and algorithms. IEEE Transactions on Signal Processing, vol. 46, no. 11, pp. 2862– 2876 (November 1998) 13. A. L. L. Ramos, J. Apolin´ario Jr, and M. G. Siqueira, A new order recursive multiple order multichannel fast QRD algorithm. Thirty-Eighth Annual Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, pp. 965–969 (November 2004) 14. C. A. Medina, J. Apolin´ario Jr., and M. G. Siqueira, A unified framework for multichannel fast QRD-LS adaptive filters based on backward prediction errors. IEEE International Midwest Symposium on Circuits and Systems, MWSCAS’02, Tulsa, USA, pp. 668–671 (August 2002) 15. A. L. L. Ramos, J. A. Apolin´ario Jr., and S. Werner, Multichannel fast QRD-LS adaptive filtering: block-channel and sequential-channel algorithms based on updating backward prediction errors. Signal Processing, vol. 87, pp. 1781–1798 (July 2007) 16. M. Shoaib, S. Werner, J. A. Apolin´ario Jr., and T. I. Laakso, Multichannel fast QRdecomposition RLS algorithms with explicit weight extraction. European Signal Processing Conference, EUSIPCO’2006, Florence, Italy (September 2006)

11 Weight Extraction of Fast QRD-RLS Algorithms

315

UN DE R

RE VI EW

17. Changsoo Eun and Edward J. Powers, A new Volterra predistorter based on the indirect learning architecture. IEEE Transactions on Signal Processing, vol. 45, no. 1, pp. 20–30 (January 1997) 18. M. Bouchard, Multichannel recursive-least-squares algorithms and fast-transversal-filter algorithms for active noise control and sound reproduction systems. IEEE Transactions on Speech and Audio Processing, vol. 8, no. 5, pp. 606–618 (September 2000) 19. P. A. Regalia and M. G. Bellanger, On the duality between fast QR methods and lattice methods in least squares adaptive filtering. IEEE Transactions on Signal Processing, vol. 39, no. 4, pp. 876–891 (April 1991) 20. M. D. Miranda and M. Gerken, A hybrid least squares QR-lattice algorithm using a priori errors. IEEE Transactions on Signal Processing, vol. 45, no. 12, pp. 2900–2911 (December 1997) 21. Ali H. Sayed, Fundamentals of Adaptive Filtering. John Wiley & Sons, Inc., Hoboken, New Jersey, USA (2003) 22. M. Shoaib, S. Werner, and J. A. Apolin´ario Jr., Reduced complexity solution for weight extraction in QRD-LSL algorithm. IEEE Signal Processing Letters, vol. 15, pp. 277–280 (2008)

RE VI EW

Index

accumulated quantization error, 233 active noise control, 307 adaptive equalization, 182 adaptive filter applications, 5 basic configuration, 5 concept of an, 4 main characteristics, 7 adaptive prewhitening, 181 Affine Projection Algorithm, 20 algorithm stability, 277 ANC, see active noise control annihilation-reording look-ahead, 264 APA, see Affine Projection Algorithm application(s), 135, 139 example, 137, 164, 165

UN DE R

back-substitution, 66 backward error analysis, 199 backward prediction, 54 error, 136, 163, 167 matrix, 151 vector, 168 problem, 151, 152, 156, 157 backward stability, 200 beamforming adaptive beamformer, 333 BIBO, see Bounded Input Bounded Output binormalized data-reusing LMS algorithm, 17, 20 block processing realization, 272 block updating, 183 BNDR-LMS, see binormalized data-reusing LMS algorithm bounded distance, 207 Bounded Input Bounded Output, 274 burst-trained equalizer, 300

channel equalization, 7 Cholesky factor, 38, 176 inverse, 180 coarse-grain pipelining, 262 computational complexity, 167, 186, 282, 313 concurrent algorithm, 263 condition number, 14, 26 controllability matrix, 209 conventional QRD-RLS algorithm, 45, 49, 228 dynamic range of internal variables, 231 conventional RLS algorithm, 24 convergence factor, 12 conversion factor, 54 Coordinate Rotation Digital Computer, 260 CORDIC, see Coordinate Rotation Digital Computer arithmetic, 259 cost function, 36 Data-reusing LMS algorithm, 16 decoupling property, 104 exact decoupling of the LSL predictor, 110 deterministic cross-correlation vector, 37 distributed weight extraction, 305 downdate, 190 DR-LMS, see Data-reusing LMS algorithm equalizer, 301 equicontinuity, 203 equivalent-output filtering, 302 error accumulation, 222 error feedback mechanism, 112, 132 error propagation, 235 Euclidean norm, 196 Euclidean space, 40 exact initialization, 50

341

342

Index computational complexity, 88 FTF, see fast transversal filter Generalized Sidelobe Canceler, 324 Givens rotation, 42, 111, 139, 145, 148, 154, 173, 261, 264, 276, 284 complex version, 69 matrices, 44, 148 Gram-Schmidt procedure, 57, 60, 110 GSC, see Generalized Sidelobe Canceler

UN DE R

RE VI EW

fast algorithms, 76, 210, 293 Fast QRD-Lattice algorithm, 251 finite precision analysis, 253 infinite precision analysis, 248 Fast QRD-RLS algorithms based on a posteriori backward prediction errors, 83 Fast QRD-RLS algorithms based on a posteriori forward prediction errors, 79 Fast QRD-RLS algorithms based on a priori backward prediction errors, 86 Fast QRD-RLS algorithms based on a priori forward prediction errors, 80 fast transversal filter, 106, 130 stabilized FTF algorithm, 132 filter information, 181 Potter’s square-root, 181 square-root covariance, 181 filtered-x RLS algorithm, 307 fine-grain pipelining, 259, 263, 276 finite precision analysis of the QRD-RLS algorithm, 226 finite-precision, 274, 277 environment, 129 forgetting factor, 11 forward prediction, 57 error, 147, 155 covariance matrix, 154 matrix, 154 vector, 155 error matrix, 152 problem, 142, 156, 157 problems, 151, 161 forward stability, 199 forward substitution, 66 FQR POS B, see Fast QRD-RLS algorithms based on a posteriori backward prediction errors matrix equations, 86 FQR POS F, see Fast QRD-RLS algorithms based on a posteriori forward prediction errors matrix equations, 80 FQR PRI B, see Fast QRD-RLS algorithms based on a priori backward prediction errors matrix equations, 87 FQR PRI F, see Fast QRD-RLS algorithms based on a priori forward prediction errors matrix equations, 81 FQRD-RLS, see QRD-RLS, fast algorithms classification of algorithms, 76

Hankel matrix, 212 Householder matrix, 172 transforms, 171 hyperbolic, 174 row, 175 hyperbolic norm, 174 hypernormal matrix, 174

IBP, see (intermediate backward) prediction error IFP, see (intermediate forward) prediction error infinite precision analysis of the QRD-RLS algorithm, 227 initialization procedure, 49 input data matrix, 36 triangularization of the, 37 input vector, 135, 137, 139, 140 input-data deterministic autocorrelation matrix, 37 Instantaneous Square Error, 10 interference cancelation, 7 interpolation, 105, 106 error, 106, 110 modified QR-decomposition for, 114 Inverse QRD-RLS algorithm, 63, 65, 132 Constrained IQRD-RLS algorithm, 326 GSC-based IQRD-RLS algorithm, 330 IQRD, see Inverse QRD-RLS algorithm ISE, see Instantaneous Square Error joint-process estimation, 47, 115, 127 KaGE RLS algorithm, 132 Kalman filtering, 181 gain, 176 gain vector, 127 Lagrange multipliers, 18 lattice filter, 104 LCAF, see linearly constrained adaptive filter LCMV, see linearly constrained minimum variance Least-Mean Square algorithm, 14

Index

343

matrix inversion lemma, 23 McWhirter structure, 50 Mean-Square Error, 9 misadjustment, 15, 25 modified filtered-x FQRD-RLS, 309, 311 structure, 309 MSE, see Mean-Square Error

backward, 110 intermediate, 109, 110 forward, 110 intermediate, 109, 110 problem conditioning, 199 projection operator, 39 pseudo-inverse, 197

RE VI EW

least-squares, 104 least-squares lattice, 104 linear interpolation, 105, 106 linear prediction, 105 linearly constrained adaptive filter, 318 linearly constrained minimum variance, 317 LMS, see Least-Mean Square algorithm lookahead techniques, 263 lower triangular matrix, 47 lower triangularization algorithms (updating backward prediction errors), 82 LS, see least-squares LSL, see least-squares lattice interpolator, 106–108 orthogonal bases for LSL interpolator, 109 predictor, 106, 110, 111

NDR-LMS, see Normalized Data-reusing algorithm NLMS, see Normalized LMS algorithm nonstationary environment, 26 Normalized Data-reusing algorithm, 16 Normalized LMS algorithm, 14 numerical robustness, 173, 179 numerical stability, 195, 331

UN DE R

optimal solution, 37, 39 order recursive FQRD-RLS algorithms, 87 order-recursive, 163 implementations, 137 MC-FQRD-RLS algorithms, 161 property, 104 orthogonal triangularization, 42 orthogonal-projections algorithm, 17 orthogonality principle, 39 overflow, 233 parallelism, 269 persistence of excitation, 37, 207 perturbation, 201 pipelinability, 263 pipelined implementation, 259 pipelined QRD-RLS adaptive filter, 274, 279 pipelined realization, 271 prediction error, 109

QR decomposition, 23, 104, 292 QR-decomposition-based least-squares lattice algorithm, 104 QRD, see QR decomposition QRD lattice algorithm, 220 QRD-based fast Kalman algorithm, 104 QRD-LSL, see QR-decomposition-based least-squares lattice algorithm QRD-LSL prediction, 115 QRD-LSL prediction algorithm, 114 QRD-RLS fast algorithms, 75 Householder block exact inverse QRD-RLS algorithm, 186 Householder block exact QRD-RLS algorithm, 183 quantization effects, 37 error, 233

Recursive Least Squares algorithm, 176 Recursive Least-Squares algorithm, 22, 104 reflection coefficients, 104 regression coefficients, 124 retiming technique, 268 RLS, see Recursive Least-Squares algorithm, see Recursive Least Squares Householder RLS algorithm, 176 sliding window, 190 rotated vector, 40 rotating mode, 261 rotation matrix, 42 sequential algorithm, 263 SFG, see signal flow graph signal flow graph, 261 signal prediction, 8 Singular Value Decomposition, 197 soft-start procedure, 52 spectral analysis, 329 spectral norm, 196 square-root factor, 176 square-root-free, 103 QRD-RLS algorithm, 67 SRF, see square-root-free (QRD-LSL)-based RLS algorithm, 127

344

Index

UN DE R

Toeplitz matrix, 212 tracking performance, 26

transversal filters, 104 upper triangular matrix, 47 upper triangularization algorithms (updating forward prediction errors), 77

RE VI EW

Givens rotation, 112 Givens rotation with a feedback mechanism, 111, 113 QRD-LSL interpolation algorithm, 113, 131 QRD-LSL prediction algorithm, 124 stability, 30 analysis, 232 state transition matrix, 209 state vector, 217 steepest-descent algorithm, 13 step-size, 12 SVD, see Singular Value Decomposition system identification, 6, 295 systolic architecture, 260 systolic array, 262

vector norm, 196 vectoring mode, 261

weight extraction, 262, 291 explicit, 262 implicit, 262 in FQRD-RLS algorithm, 296 Weighted Least-Squares, 10, 35 Wiener solution, 10 WLS, see Weighted Least-Squares, see Weighted Least-Squares