Partial Differentiation and Stationary Points

Comp.145  February 1997  Dr. A.J. Mestel

Let f(x, y) be a function of two independent variables. If we hold y constant, then we can treat f as a function of x alone, and differentiate it. This process is called partial differentiation, and the result is called the partial derivative of f with respect to x at constant y. We may similarly regard x as constant and differentiate f with respect to y. These partial derivatives are written

\frac{∂f}{∂x}  or \frac{∂f}{∂x} |_{y}  and \frac{∂f}{∂y}  or \frac{∂f}{∂y} |_{x} .

Often, it is obvious what is being kept constant during partial differentiation, but if there is any doubt, it is safest to write it explicitly. Sometimes a suffix notation is used to denote partial derivatives, so that f_{x} ≡ \frac{∂f}{∂x} and so on. We can also find the second (or even higher) partial derivatives

f_{xx} ≡ \frac{∂^{2}f}{∂x^{2}} ≡ \frac{∂}{∂x} (\frac{∂f}{∂x} ) f_{yy} ≡ \frac{∂^{2}f}{∂y^{2}} ≡ \frac{∂}{∂y} (\frac{∂f}{∂y} ) f_{xy} ≡ \frac{∂^{2}f}{∂x∂y} ≡ \frac{∂}{∂x} (\frac{∂f}{∂y} ) .

An important result is that f_{xy} = f_{yx}. As the two differentiations are independent, it does not matter in which order we perform then.

Example: Suppose f(x, y) = y sin x + xy. Then by calculation,

f_{x} = y cos x + y, f_{y} = sin x + x, f_{xx} = −y sin x, f_{xy} = cos x + 1 = f_{yx}, f_{yy} = 0 .

If x and y vary with a third variable, t, we can differentiate f(x, y) with respect to t using the chain rule,

\frac{df}{dt} = \frac{∂f}{∂x} \frac{dx}{dt} + \frac{∂f}{∂y} \frac{dy}{dt} .

This essentially states that as x and y are varying independently, the change in f is the sum of the changes due to x varying and y varying. Thus if x and y change by small amounts δx and δy, the amount δf by which f changes is approximately

δf ≃ δxf_{x} + δyf_{y} .

Be careful to distinguish between dx, δx and ∂x.

Similar results hold for functions of three or more variables. For the rest of this sheet we will consider functions of three variables, f(x, y, z).

The vector of partial derivatives is called “grad f” and denoted by ∇f,

∇f ≡ (\frac{∂f}{∂x} , \frac{∂f}{∂y} , \frac{∂f}{∂z} )^{T} so that δf ≃ ∇f · (δx, δy, δz) .

If the vector (δx, δy, δz) is such that the value of f does not change, then δf = 0. It follows that the vector ∇f is perpendicular to the surface defined by f(x, y, z) = constant. It is also possible to use the scalar and vector products to define the “divergence” and “curl” of a vector \underline{A}, written ∇ · \underline{A} and ∇ ∧ \underline{A}. These are beyond the scope of this course.

Just as the first partial derivatives form a vector, we can write all the second partial derivatives as a matrix,

H = ( \begin{matrix}f_{xx} & f_{xy} & f_{xz} \\ f_{yx} & f_{yy} & f_{yz} \\ f_{zx} & f_{zy} & f_{zz} \end{matrix} ) .

Since f_{xy} = f_{yx} and so on, this matrix is symmetric, and hence has real eigen-values. These eigenvalues are important when considering:

Stationary points of functions of more than one variable.

If we want to find a minimum of a function of one variable, say g(x), we look for a value of x, say x = x_{0}, for which the derivative is zero, g′(x_{0}) = 0. One way of the determining the nature of stationary point x = x_{0} is to look at the second derivative evaluated at the point, g′′(x_{0}). If this g′′(x_{0}) > 0 we have a minimum, if g′′(x_{0}) < 0 we have a maximum, while if g′′(x_{0}) = 0, we don’t know. The reason for this is that if we move a small distance ε from x_{0}, using a Taylor series g(x_{0} + ε) = g(x_{0}) + εg′(x_{0}) + \frac{1}{2}ε^{2}g′′(x_{0}) + O(ε^{3}). Thus g(x_{0} + ε) > g(x_{0}) if g′′(x_{0}) > 0, so that x_{0} is a minimum.

Consider now a function of three variables, f(x, y, z). A point (x, y, z) = (x_{0}, y_{0}, z_{0}) is stationary if the value of f is unchanged to leading order when x_{0} y_{0} and z_{0} are altered by arbitrary, independent, small amounts. Equivalently, the value of f must remain stationary no matter in which direction we choose to move away from that point. This requires that simultaneously,

\frac{∂f}{∂x} = 0  and  \frac{∂f}{∂y} = 0  and  \frac{∂f}{∂z} = 0 .

In order to determine the nature of the stationary point, we look at the second derivatives evaluated at that point, just as for the function of one variable g(x). We evaluate the matrix of second partial derivatives at that point, to obtain H_{0} = H(x_{0}, y_{0}, z_{0}). The equivalent criteria for maxima and minima are the following:

(a) If all the eigenvalues of the matrix H_{0} are strictly positive, then the point is a minimum.

(b) If all the eigenvalues of the matrix H_{0} are strictly negative, then the point is a maximum.

(c) If at least one eigenvalue is strictly positive and at least one is strictly negative, then the point is a saddle point. This means the value of the function increases in some direction and decreases in another.

(d) If one of the eigenvalues is zero and the others are the same sign, we don’t know.