Cambridge University Press 978-1-107-04316-9 - Mathematical Foundations of Infinite-Dimensional Statistical Models Evarist Giné and Richard Nickl Table of Contents More information

Contents

page xi

Preface 1 1.1

Nonparametric Statistical Models Statistical Sampling Models 1.1.1 Nonparametric Models for Probability Measures 1.1.2 Indirect Observations

1.2

Gaussian Models 1.2.1 Basic Ideas of Regression 1.2.2 Some Nonparametric Gaussian Models 1.2.3 Equivalence of Statistical Experiments

1.3

Notes

2 2.1

Gaussian Processes Definitions, Separability, 0-1 Law, Concentration 2.1.1 Stochastic Processes: Preliminaries and Definitions 2.1.2 Gaussian Processes: Introduction and First Properties

2.2

Isoperimetric Inequalities with Applications to Concentration 2.2.1 The Isoperimetric Inequality on the Sphere 2.2.2 The Gaussian Isoperimetric Inequality for the Standard Gaussian Measure on RN 2.2.3 Application to Gaussian Concentration

2.3 2.4

The Metric Entropy Bound for Suprema of Sub-Gaussian Processes Anderson’s Lemma, Comparison and Sudakov’s Lower Bound 2.4.1 Anderson’s Lemma 2.4.2 Slepian’s Lemma and Sudakov’s Minorisation

2.5

The Log-Sobolev Inequality and Further Concentration 2.5.1 Some Properties of Entropy: Variational Definition and Tensorisation 2.5.2 A First Instance of the Herbst (or Entropy) Method: Concentration of the Norm of a Gaussian Variable about Its Expectation

2.6

Reproducing Kernel Hilbert Spaces 2.6.1 Definition and Basic Properties 2.6.2 Some Applications of RKHS: Isoperimetric Inequality, Equivalence and Singularity, Small Ball Estimates 2.6.3 An Example: RKHS and Lower Bounds for Small Ball Probabilities of Integrated Brownian Motion

2.7 2.8

Asymptotics for Extremes of Stationary Gaussian Processes Notes

1 2 2 3 4 4 6 8 13 15 15 15 19 26 26 30 32 36 48 48 52 60 60 62 66 66 72 79 88 102

vii

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-04316-9 - Mathematical Foundations of Infinite-Dimensional Statistical Models Evarist Giné and Richard Nickl Table of Contents More information

viii

Contents 3 3.1

Empirical Processes Definitions, Overview and Some Background Inequalities 3.1.1 Definitions and Overview 3.1.2 Exponential and Maximal Inequalities for Sums of Independent Centred and Bounded Real Random Variables 3.1.3 The L´evy and Hoffmann-Jørgensen Inequalities 3.1.4 Symmetrisation, Randomisation, Contraction

3.2

Rademacher Processes 3.2.1 A Comparison Principle for Rademacher Processes 3.2.2 Convex Distance Concentration and Rademacher Processes 3.2.3 A Lower Bound for the Expected Supremum of a Rademacher Process

3.3

The Entropy Method and Talagrand’s Inequality 3.3.1 The Subadditivity Property of the Empirical Process 3.3.2 Differential Inequalities and Bounds for Laplace Transforms of Subadditive Functions and Centred Empirical Processes, λ ≥ 0 3.3.3 Differential Inequalities and Bounds for Laplace Transforms of Centred Empirical Processes, λ < 0 3.3.4 The Entropy Method for Random Variables with Bounded Differences and for Self-Bounding Random Variables 3.3.5 The Upper Tail in Talagrand’s Inequality for Nonidentically Distributed Random Variables*

3.4

First Applications of Talagrand’s Inequality 3.4.1 Moment Inequalities 3.4.2 Data-Driven Inequalities: Rademacher Complexities 3.4.3 A Bernstein-Type Inequality for Canonical U-statistics of Order 2

3.5

Metric Entropy Bounds for Suprema of Empirical Processes 3.5.1 Random Entropy Bounds via Randomisation 3.5.2 Bracketing I: An Expectation Bound 3.5.3 Bracketing II: An Exponential Bound for Empirical Processes over Not Necessarily Bounded Classes of Functions

3.6

ˇ Vapnik-Cervonenkis Classes of Sets and Functions ˇ 3.6.1 Vapnik-Cervonenkis Classes of Sets 3.6.2 VC Subgraph Classes of Functions 3.6.3 VC Hull and VC Major Classes of Functions

3.7

Limit Theorems for Empirical Processes 3.7.1 3.7.2 3.7.3 3.7.4

Some Measurability Uniform Laws of Large Numbers (Glivenko-Cantelli Theorems) Convergence in Law of Bounded Processes Central Limit Theorems for Empirical Processes I: Definition and Some Properties of Donsker Classes of Functions 3.7.5 Central Limit Theorems for Empirical Processes II: Metric and Bracketing Entropy Sufficient Conditions for the Donsker Property 3.7.6 Central Limit Theorems for Empirical Processes III: Limit Theorems Uniform in P and Limit Theorems for P-Pre-Gaussian Classes

3.8

Notes

© in this web service Cambridge University Press

109 109 109 113 121 127 135 136 139 144 149 149 153 158 161 165 171 171 173 175 184 184 195 206 212 212 217 222 228 229 233 242 250

257

261 286

www.cambridge.org

Cambridge University Press 978-1-107-04316-9 - Mathematical Foundations of Infinite-Dimensional Statistical Models Evarist Giné and Richard Nickl Table of Contents More information

Contents 4 4.1

Function Spaces and Approximation Theory Definitions and Basic Approximation Theory 4.1.1 4.1.2 4.1.3 4.1.4

4.2

Notation and Preliminaries Approximate Identities Approximation in Sobolev Spaces by General Integral Operators Littlewood-Paley Decomposition

Orthonormal Wavelet Bases 4.2.1 Multiresolution Analysis of L2 4.2.2 Approximation with Periodic Kernels 4.2.3 Construction of Scaling Functions

4.3

Besov Spaces 4.3.1 4.3.2 4.3.3 4.3.4 4.3.5 4.3.6 4.3.7

4.4

Definitions and Characterisations Basic Theory of the Spaces Bspq Relationships to Classical Function Spaces Periodic Besov Spaces on [0, 1] Boundary-Corrected Wavelet Bases* Besov Spaces on Subsets of Rd Metric Entropy Estimates

Gaussian and Empirical Processes in Besov Spaces 4.4.1 Random Gaussian Wavelet Series in Besov Spaces 4.4.2 Donsker Properties of Balls in Besov Spaces

4.5

Notes

5 5.1

Linear Nonparametric Estimators Kernel and Projection-Type Estimators 5.1.1 Moment Bounds 5.1.2 Exponential Inequalities, Higher Moments and Almost-Sure Limit Theorems 5.1.3 A Distributional Limit Theorem for Uniform Deviations*

5.2

Weak and Multiscale Metrics 5.2.1 Smoothed Empirical Processes 5.2.2 Multiscale Spaces

5.3

Some Further Topics 5.3.1 Estimation of Functionals 5.3.2 Deconvolution

5.4

Notes

6 6.1

The Minimax Paradigm Likelihoods and Information 6.1.1 Infinite-Dimensional Gaussian Likelihoods 6.1.2 Basic Information Theory

6.2

Testing Nonparametric Hypotheses 6.2.1 6.2.2 6.2.3 6.2.4

6.3

Construction of Tests for Simple Hypotheses Minimax Testing of Uniformity on [0, 1] Minimax Signal-Detection Problems in Gaussian White Noise Composite Testing Problems

Nonparametric Estimation 6.3.1 Minimax Lower Bounds via Multiple Hypothesis Testing

© in this web service Cambridge University Press

ix 291 291 291 295 301 304 305 305 312 316 327 327 338 347 352 361 366 372 379 379 381 386 389 389 391 405 411 421 421 434 439 439 451 462 467 467 468 473 476 478 485 492 494 511 512

www.cambridge.org

Cambridge University Press 978-1-107-04316-9 - Mathematical Foundations of Infinite-Dimensional Statistical Models Evarist Giné and Richard Nickl Table of Contents More information

x

Contents 6.3.2 Function Estimation in L∞ Loss 6.3.3 Function Estimation in Lp -Loss

6.4

Nonparametric Confidence Sets 6.4.1 Honest Minimax Confidence Sets 6.4.2 Confidence Sets for Nonparametric Estimators

6.5

Notes

7 7.1 7.2

Likelihood-Based Procedures Nonparametric Testing in Hellinger Distance Nonparametric Maximum Likelihood Estimators 7.2.1 7.2.2 7.2.3 7.2.4

7.3

Rates of Convergence in Hellinger Distance The Information Geometry of the Likelihood Function The Maximum Likelihood Estimator over a Sobolev Ball The Maximum Likelihood Estimator of a Monotone Density

Nonparametric Bayes Procedures 7.3.1 7.3.2 7.3.3 7.3.4

General Contraction Results for Posterior Distributions Contraction Results with Gaussian Priors Product Priors in Gaussian Regression Nonparametric Bernstein–von Mises Theorems

7.4

Notes

8 8.1

Adaptive Inference Adaptive Multiple-Testing Problems

515 518 522 523 524 537 541 542 546 547 551 554 563 570 573 578 582 591 603

Notes

607 607 608 612 614 614 620 628 629 638 644 657 664

References Author Index Index

667 683 687

8.1.1 Adaptive Testing with L2 -Alternatives 8.1.2 Adaptive Plug-in Tests for L∞ -Alternatives

8.2

Adaptive Estimation 8.2.1 Adaptive Estimation in L2 8.2.2 Adaptive Estimation in L∞

8.3

Adaptive Confidence Sets 8.3.1 8.3.2 8.3.3 8.3.4

8.4

Confidence Sets in Two-Class Adaptation Problems Confidence Sets for Adaptive Estimators I Confidence Sets for Adaptive Estimators II: Self-Similar Functions Some Theory for Self-Similar Functions

© in this web service Cambridge University Press

www.cambridge.org