This book is designed to help problem solvers make the best use of optimization software--i.e., to use existing methods most effectively whenever possible, and to adapt and modify techniques for particular problems if necessary. The contents of this book therefore include some topics that are essential for all those who wish to solve optimization problems. In addition, certain topics are treated that should be of special interest in most practical optimization problems. For example, advice is given to users who wish to take an active role in formulating their problems so as to enhance the chances of solving them successfully, and to users who need to understand why a certain method fails to solve their problem.
Introduction: Definition of Optimization Problems. Classification of Optimization Problems. Overview of Topics. Fundamentals: Introduction to Errors in Numerical Computation. Introduction to Numerical Linear Algebra. Linear Equations. Matrix Factorizations. Elements of Multivariate Analysis. Optimality Conditions: Characterization of a Minimum. Unconstrained Optimization. Linearly Constrained Optimization. Nonlinearly Constrained Optimization. Unconstrained Methods: Methods for Univariate Functions. Methods for Multivariate Non-Smooth Functions. Methods for Multivariate Smooth Functions. Second Derivative Methods. First Derivative Methods. Non-Derivative Methods for Smooth Functions. Methods for Sums of Squares. Methods for Large-Scale Problems. Linear Constraints: Methods for Linear Equality Constraints. Active Set Methods for Linear Inequality Constraints. Special Problem Categories. Problems with Few General Linear Constraints. Special Forms of the Constraints. Large-Scale Linearly Constrained Optimization. Finding an Initial Feasible Point. Implementation of Active Set Methods. Nonlinear Constraints: The Formulation of Algorithms. Penalty and Barrier Function Methods. Reduced-Gradient and Gradient-Projection Methods. Augmented Lagrangian Methods. Projected Lagrangian Methods. Lagrange Multiplier Estimates. Large-Scale Nonlinearly Constrained Optimization. Special Problem Categories. Modelling: Introduction. Classification of Optimization Problems. Avoiding Unnecessary Discontinuities. Problem Transformations. Scaling. Formulation of Constraints. Problems with Discrete or Integer Variables. Practicalities: Use of Software. Properties of the Computed Solution. Assessment of Results. What Can Go Wrong (and what to do about it). Estimating the Accuracy of the Problem Functions. Computing Finite Differences. More About Scaling. Questions and Answers. Bibliography. Index.