Linear approximations

When we say that f is differentiable at a we mean that

$$\frac{f(a+h)-f(a)}{h}$$

has a limit as h approaches , or equivalently,

$$\frac{f(x)-f(a)}{x-a}$$

has a limit as x approaches a. We have on many occasions said that we should regard the value of this limit as the slope, m, of a line tangent to the graph of y = f(x) at the point (a,f(a)). This tangent line has equation y = f(a) + m(x-a). If we write this out symbolically we have

$$\lim_{x\rightarrow a}\frac{f(x)-f(a)}{x-a} = m.$$

By using various properties of limits, we can rewrite this last equation as

$$\lim_{x\rightarrow a}\frac{\vert f(x) - (f(a) + m(x-a))\vert}{\vert x-a\vert} = 0$$

We should interpret this new equation as saying that f has a derivative at a if we can find some line with equation y = f(a) + m(x-a) so that

$$\lim_{x\rightarrow a}\frac{\vert f(x) - (f(a) + m(x-a))\vert}{\vert x-a\vert} = 0.$$

Notice that the numerator of this quotient gives the magnitude of the difference between the second coordinate of (x,f(x) and the second coordinate (x,f(a)+m(x-a)), while the denominator gives the magnitude of the difference between the first coordinates. One interpretation, then, of the numerator is that it tells us how well f(a) + m(x-a) approximates f(x). Thus, f has a derivative at a means that there is some line which approximates f so well that the error in the approximation, |f(x) - (f(a) + m(x-a))| goes to faster than |x-a| goes to , and we call the slope of this line the derivative of f at a. We denote this special slope by f'(a). Furthermore, no other slope will have this property. For this reason, the function L(x) = f(a) + f'(a)(x-a) is called the best linear approximation to f at a. For example, the best linear approximation to $\arctan$ at 1 is $L(x) = \arctan(1) + \arctan'(1)(x-1) = (\pi/4) + (1/2)(x-1)$.

An approximation is not very valuable if we have no way to assess its accuracy. It is evident to the most untutored eye that a tangent line is close to a graph near the point of tangency, but if we are to make qualitative statements about particular values we must have more information. If we are to make these statements we must have more information about f. All we know so far is that the error appoaches faster than (x-a) approaches . We will now show that under certain circumstances, the error approaches like (x-a)².

Suppose, for example, that we knew that f has a second derivative at a. Since the second derivative is the derivative of the first derivative, f must have a first derivative not only at a, but in an interval containing a, since the second derivative of f at a is the limit of

$$\frac{f'(x)-f'(a)}{x-a}$$

as x approaches a. Let us denote this second derivative in the usual fashion by f''(a), so we may write

$$\lim_{x\rightarrow a} \frac{f'(x)-f'(a)}{x-a} = f''(a)$$

Consider now the quotient

$$\frac{f(x) - f(a) - f'(a)(x-a)}{(x-a)^2}$$

Since both the numerator and denominator of this quotient approach , it is reasonable to try to apply L'Hopital's Rule to see if the quotient itself has a limit as x approaches a. The quotient of the derivatives is

$$\frac{f'(x) - 0 - f'(a)(1-0)}{2(x-a)} = \frac{1}{2}\frac{f'(x)-f'(a)}{x-a}$$

which, by the defintion of f''(a) has a limit of f''(a)/2. Hence, by L'Hopital's Rule, we may conclude that

$$\lim{x\rightarrow a}\frac{f(x) - f(a) - f'(a)(x-a)}{(x-a)^2} = \frac{f''(a)}{2}.$$

This tells us that

$$\vert f(x)-(f(a)+f'(a)(x-a)\vert\approx \frac{f''(a)}{2}(x-a)^2\;\;{\rm when}\;\:\vert x-a\vert<<1.$$

While this is an improvement, it does not tell us what happens for a specific value of x. To do this, we need different information about f.

So, pick a particular value for x. We know that there is some number, call it B, so that

f(x) - f(a) - f'(a)(x-a) = B(x-a)²,

or

f(x) - f(a) - f'(a)(x-a) - B(x-a)² = 0.

The goal is to express B in terms of f''. To this end, we are going to invent a rather strange function to which we will apply Rolle's theorem. Define $R:[0,1]\rightarrow(-\infty,\infty)$ by

R(t) = f(tx + (1-t)a) -f(a) - f'(a)(x-a)t - B(x-a)²t²

Where did this come from? First looked for the linear function of t which was equal to a when t=0 and x when t=1. This function has rule tx + (1-t)a. We then replaced x with tx + (1-t)a in f(x) - f(a) - f'(a)(x-a) - B(x-a)². If f is nice enough, this will give a function to which we can easily apply Rolle's Theorem.

It is easy to see that

$$\begin{array} {rcl} R(0) & = & f(a) - f(a) - f'(a)(x-a)\cdot... ...& = & f(x) - f(a) - f'(a)(x-a) - B(x-a)^2\\ & = & 0\end{array}$$

where the last equality is true from the definition of B. We see from the chain rule that R will be differentiable on (0,1) if f is differentiable between x and a. Supposing that this is so, we have for some $s\in(0,1)$,R'(s) = 0. What does R'(t) look like?

$$\begin{array} {rcl} R'(t) & = & f'(tx + (1-t)a)(tx + (1-t)a... ... = & (x-a)f'(tx + (1-t)a) - f'(a)(x-a) - 2B(x-a)^2t.\end{array}$$

Well, not only is R'(s) = 0 for some $s\in(0,1)$, we can also see that

$$\begin{array} {rcl} R'(0) & = & f'(a)(x-a) - f'(a)(x-a) - 2B(x-a)\cdot 0\\ & = & 0.\end{array}$$

So, if R' is differentiable on (0,s), we may apply Rolle's Theorem to R' and get for some $r\in(0,s)\subset(0,1)$ that R''(r) = 0. What does R''(t) look like?

$$\begin{array} {rcl} R''(t) & = & (x-a)f''(tx + (1-t)a)(tx +... ...) - 2B(x-a)^2\\ & = & (x-a)^2(f''(tx + (1-t)a) - 2B)\end{array}$$

so R''(r) = 0 gives the equation

0 = (x-a)²(f''(rt +(1-r)a) - 2B)

so presuming that x and a are different, and setting x_r = rx + (1-r)a,

f''(x_r) - 2B = 0

or

$$B = \frac{1}{2}f''(x_r).$$

Note that x_r is some number lying strictly bewteen x and a. We have derived the following, a special case of Taylor's Theorem:

Theorem 8513

Suppose that f is twice differentiable on the open interval with endpoints x and a, continuous at a and x, and that f' is continuous at a. Then for some number x_r lying strictly between a and x,

$$f(x) - (f(a) - f'(a)(x-a)) = \frac{f''(x_r)}{2}(x-a)^2.$$

This says we can get precise information of the accuracy of the linear approximation at a if we can say how large or small f'' is between x and a.

Here are three examples.

How well does that linear approximation to $\sin$ at approximate $\sin(0.5)$? The linear approximation to $\sin$ at is $L(x) = sin(0) + \sin'(0)(x-0) = x$. What we want is an estimate of $\vert\sin(0.5) - 0.5\vert$. What we know is that for some $x_r\in (0,0.5)$ that

$$\sin(0.5) - 0.5 = \frac{-\sin(x_r)}{2}(0.5)^2$$

so

$$\begin{array} {rcl} \vert\sin(0.5) - 0.5\vert & = & {\displ... ...playstyle \frac{sin(\pi/6)}{2}(0.25)}\\ & = & 0.0625\end{array}$$

since $\pi/6 \gt 0.5$ and $\vert-\sin\vert$ is increasing on $(0,\pi/2)$.

How well does the linear approximation to $\ln$ at 1 approximate $\ln(0.9)$? The linear approximation to $\ln$ at 1 is $L(x) = \ln(1) + \ln'(1)(x-1) = x-1$. What we want is an estimate of $\vert\ln(0.9) - (0.9-1)\vert = \vert\ln(0.9)-(-0.1)\vert$.What we know is that for some $x_r\in (0.9,1)$ that

$$\ln(0.9) -(-0.1) = \frac{\\ ln''(x_r)}{2}(-0.1)^2 = \frac{-1}{2x_r^2}(0.01)$$

so

$$\begin{array} {rcl} \vert\ln(0.9) - (-0.1)\vert & = & {\dis... ...2\;{\rm is\;decreasing\;on\;}90.0,1)}\\ & = & 1/162.\end{array}$$

How well does the linear approximation to $\arctan$ at 1 approximate $\arctan(1.1)$? The linear approximation to $\arctan$ at 1 is $L(x) = \arctan(1) + \arctan'(1)(x-1) = \pi/4 + (1/2)(x-1)$. What we want is an estimate of $\vert\arctan(1.1) - (\pi/4 + (1/2)(1.1-1)\vert = \vert\arctan(1.1)-\pi/4 - 1/20\vert$. What we know is that for some $x_r\in (1,1.1)$ that

$$\arctan(1.1)-\pi/4 - 1/20 = \frac{\arctan''(x_r)}{2}(0.1)^2 = -\frac{2x_r}{2(1+x_r^2)}(0.01)$$

so

$$\begin{array} {rcl} \vert\arctan(1.1)-\pi/4 - 1/20\vert & =... ...& \leq & \frac{1.1}{(1+1^2)^2}(0.01)\\ & = & 0.00275\end{array}$$

Two additional notes.

• If f'(a) = 0 and f'' is non-negative in some interval around a then f(a) is the smallest value of f in this interval, since for x in this interval,

  $$\begin{array} {rcl} f(x) & = & {\displaystyle f(a) + f'(a) +... ...le f(a) + \frac{f''(x_r)}{2}(x-a)^2}\\ & \geq & f(a)\end{array}$$

  Similarly, if f'' is non-positive in some interval, and f'(a) = 0, then f(a) is the largest value of f in this interval, since for x in this interval,

  $$\begin{array} {rcl} f(x) & = & {\displaystyle f(a) + f'(a) +... ... & f(a) + \frac{f''(x_r)}{2}(x-a)^2\\ & \leq & f(a).\end{array}$$

• It is possible to get other polynomial approximations. The most popular is the quadratic approximation at a: Q(x) = f(a) + f'(a)(x-a) + f''(a)(x-a)/2. With the proper conditions on f we can show (by the same method as for the linear approximation) that for some x_r between x and a that f(x) - Q(x) = f'''(x_r)(x-a)³/6.