Maximum Penalized Likelihood Estimation: Regression

This is the second volume of a text on the theory and practice of maximum penalized likelihood estimation. It is intended for graduate students in statistics, operations research and applied mathematics, as well as for researchers and practitioners in the field. The present volume deals with nonparametric regression. The emphasis in this volume is on smoothing splines of arbitrary order, but other estimators (kernels, local and global polynomials) pass review as well. Smoothing splines and local polynomials are studied in the context of reproducing kernel Hilbert spaces. The connection between smoothing splines and reproducing kernels is of course well-known. The new twist is that letting the innerproduct depend on the smoothing parameter opens up new possibilities. It leads to asymptotically equivalent reproducing kernel estimators (without qualifications), and thence, via uniform error bounds for kernel estimators, to uniform error bounds for smoothing splines and via strong approximations, to confidence bands for the unknown regression function. The reason for studying smoothing splines of arbitrary order is that one wants to use them for data analysis. Regarding the actual computation, the usual scheme based on spline interpolation is useful for cubic smoothing splines only. For splines of arbitrary order, the Kalman filter is the most important method, the intricacies of which are explained in full. The authors also discuss simulation results for smoothing splines and local and global polynomials for a variety of test problems as well as results on confidence bands for the unknown regression function based on undersmoothed quintic smoothing splines with remarkably good coverage probabilities.