Elements of Classical and Geometric Optimization

Contents Chapter 1 Optimization methods – A preview 1.1 Introduction 1.2 The continuous case – mathematical formulation 1.3 The discrete case – The travelling salesman problem 1.4 Basics of probability theory and random number generation 1.5 The brachistochrone problem 1.6 More on functional optimization: Hamilton’s principle 1.7 Constrained optimization problems and optimality conditions 1.8. Functional optimization and optimal control Concluding Remarks Exercises Notations References Chapter 2 Classical derivative-based methods of optimization 2.1 Introduction 2.2 Basic gradient methods 2.3 Quasi-Newton methods 2.4 Penalty function methods 2.5 Linear programming (LP) 2.6. Method of generalized reduced gradients 2.7 Method of feasible directions 2.8 Method of gradient projection Concluding remarks Exercises Notations References Chapter 3 – Classical derivative-free methods of optimization 3.1 Introduction 3.2 Direct search methods 3.3 Other direct search methods 3.4 Metaheuristics - Evolutionary methods Concluding remarks Exercises Notations References Chapter 4 Elements of Riemannian Differential Geometry and geometric methods of optimization 4.1 Introduction 4.2 Tangent vectors and tangent space on manifolds 4.3 Riemannian (geometric) version of some classical gradient methods 4.4. Statistical estimation by geometrical method of optimization 4.5. Stochastic processes, stochastic calculus and solution of SDEs 4.6. Analogy between statistical sampling and stochastic optimization 4.7. Geometric method of optimization by Riemannian Langevin dynamics Concluding remarks Exercises Notations References Chapter 5 Stochastic analysis on a manifold and more on geometric optimization methods 5.1. Introduction 5.2 Stochastic development on a manifold 5.3. Non-convex function optimization based on stochastic development 5.4. Parameter estimation by GALA Concluding remarks Notations References