Calculus：Applications in Constrained Optimization

Preface
Acknowledgment
Introduction

1 Linear Algebra (I): Vocabulary
1.1 Vector space Rn and its properties
1.2 Subspaces of Rn 
1.3 Linear independence
1.4 Basis and dimension
1.5 Inner product of Rn 
1.6 Gram-Schmidt process
1.7 Exercises for Chapter 1

2 Linear Algebra (II): Ranks
2.1 Review on matrices
2.2 Solving equations by Gaussian eliminations
2.3 Applications of Gaussian eliminations
2.4 Rank
2.5 Determinant
2.6 Inverse
2.7 Exercises for Chapter 2

3 Linear Algebra (III): Definiteness
3.1 Some special matrices
3.2 Motivation: Complete the squares
3.3 Eigenvalues and eigenvectors
3.4 Diagonalization of symmetric matrices
3.5 Definiteness
3.6 Sylvester's criterion
3.7 Connection with quadratic forms
3.8 Second derivative test and generalization
3.9 Proof of Sylvester's criterion
3.10 Exercises for Chapter 3

4 Constrained Optimization (I)
4.1 Optimization: Equality constraints
4.2 Non-degenerate constraint qualifications (NDCQ)
4.3 Worked example: Equality constraints
4.4 Optimization: Inequality constraints
4.5 NDCQ for inequality constrained problems
4.6 Proof of complementary slackness
4.7 Proof of non-negativity of multipliers
4.8 Worked example: Inequality constraints
4.9 Worked examples: Linear programming on R2
4.10 Exercises for Chapter 4

5 Constrained Optimization (II)
5.1 Optimization: Mixed constraints
5.2 Worked examples: Mixed constraints
5.3 Optimization: Minimization problems
5.4 Worked example: Minimization problems
5.5 Optimization: Kuhn-Tucker's formulation
5.6 Proof of Kuhn-Tucker's FOC
5.7 NDCQ in Kuhn-Tucker's formulation
5.8 Worked examples : Kuhn-Tucker's formulation
5.9 Exercises for Chapter 5

6 Envelope Theorems
6.1 Motivation: Linear budget constraint problem
6.2 Envelope Theorem for equality constraints
6.3 Worked example: Envelope Theorem
6.4 Applications: Shadow prices
6.5 Envelope Theorem for unconstrained problems
6.6 Envelope Theorem for inequality constraints
6.7 Applications in Economics
6.8 Proofs of Envelope Theorems
6.9 Exercises for Chapter 6

7 Second Order Conditions
7.1 Motivation : Bordered Hessian matrices
7.2 Bordered Hessian matrices
7.3 Second order conditions: Statement
7.4 Worked examples: Second order conditions
7.5 Second derivative test for unconstrained problems
7.6 Sketch of the proofs of SOC
7.7 Exercises for Chapter 7

Appendix. Di!erential Calculus
A. Partial derivatives
B. Chain rule for multivariable functions
C. Elementary results of optimization in multivariable

Answers to Selected Exercises
Bibliography
Index

Preface

　　Constrained optimization is a critical and contemporary subject across many disciplines. For example, many principles in classical economical theory are developed based on optimization theory to allocate scarce resources optimally. More recently, theory in optimization has been developed rapidly to catch up with the recent trend of Artificial Intelligence and Machine Learning. The increasing demand for knowledge in optimization led the Department of Mathematics at National Taiwan University to offer an 32-hour elective course on constrained optimization (titled ‘Calculus 4 – Applications in Economics and Management’) in the spring of 2019. The course is intended for students who have just finished courses in classical multivariable (differential) calculus.

　　The aforementioned course aims to extend and refine the elementary forms of the second derivative test for functions of two variables and the method of Lagrange multipliers for problems with equality constraints, as introduced in students’ earlier courses. To give a little more detail, these ideas were further developed to handle optimization problems involving both equality and inequality constraints, to identify degenerate points arising from such constraints, to derive Envelope Theorems that capture the sensitivity of the optimal value with respect to variations in the constraints and to formulate second-order conditions for constrained optimization problems in spaces of arbitrary dimension. The course has been well-received by both students and faculty members.

　　To achieve this goal, we will require a substantial background from linear algebra which is the formal language and theory of vectors and matrices. These concepts will be introduced in the beginning. Indeed, optimization theory is one of the many instances in mathematics where (linear) algebra and calculus intersect and enrich each other. The tools of both disciplines work in harmony to deepen our understanding of constrained optimization problems.

　　Although much literature and textbooks already exist on the subject, most are either too advanced or, at the other extreme, lack sufficient mathematical rigor. This book aims to introduce the basics of optimization theory in an intuitive manner that is accessible to undergraduate students who have acquired a standard course in multivariable differential calculus while maintaining some level of mathematical rigor. The authors believe that understanding both applications and mathematical foundations of optimization theory is crucial for preparing students of this generation, correctly, for more advanced courses and future challenges.