📐 The Chain Rule in Calculus

• Key Overview & Roadmap:
  • Definition of the chain rule for composite functions
  • Intuitive explanations and real-world analogies
  • Mathematical statements in Lagrange's and Leibniz's notations
  • Advanced applications (composites of multiple functions, quotient rule, inverse functions, backpropagation)
  • Higher-order derivatives and Faà di Bruno's formula

---

💡 Core Concept & Definition

In calculus, the chain rule is a fundamental formula that expresses the derivative of the composition of two differentiable functions in terms of the derivatives of the individual functions.

More precisely, if $h = z \circ y$ is the composition such that $h(x) = z(y(x))$ for every $x$, the chain rule provides the means to find $h'(x)$.

Mathematical Notations

Notation Type	Formula	Description
Lagrange's Notation	$h'(x) = z'(y(x)) \cdot y'(x)$ or $h' = (z \circ y)' = (z' \circ y) \cdot y'$	Expresses the derivative of composite functions using prime symbols.
Leibniz's Notation	$\dfrac{dz}{dx} = \dfrac{dz}{dy} \cdot \dfrac{dy}{dx}$ or $\left.\dfrac{dz}{dx}\right	_x = \left.\dfrac{dz}{dy}\right

Note: In integration, the counterpart to the chain rule is the substitution rule.

---

🧠 Intuitive Explanation

If a car travels twice as fast as a bicycle and the bicycle is four times as fast as a walking man, then the car travels $2 \times 4 = 8$ times as fast as the man. — George F. Simmons

Understanding the Analogy

• Let $z$, $y$, and $x$ be the variable positions of the car, the bicycle, and the walking man, respectively.
• The rate of change of relative positions of the car and the bicycle is $\dfrac{dz}{dy} = 2$.
• The rate of change of relative positions of the bicycle and the walking man is $\dfrac{dy}{dx} = 4$.
• Therefore, the combined rate of change of the car relative to the walking man is: $\dfrac{dz}{dx} = \dfrac{dz}{dy} \cdot \dfrac{dy}{dx} = 2 \cdot 4 = 8$

Time-Based Rates

Because the rate of change of positions is the ratio of speeds, and speed is the derivative of position with respect to time: $\dfrac{dz}{dx} = \dfrac{\dfrac{dz}{dt}}{\dfrac{dx}{dt}} \iff \dfrac{dz}{dt} = \dfrac{dz}{dx} \cdot \dfrac{dx}{dt}$ This represents another direct application of the chain rule.

---

📝 Formal Statements

1. Single Variable Function

If $g$ is a function differentiable at a point $c$ (meaning $g'(c)$ exists) and $f$ is differentiable at $g(c)$, then the composite function $f \circ g$ is differentiable at $c$. Its derivative is given by: $(f \circ g)'(c) = f'(g(c)) \cdot g'(c)$

In abbreviated form: $(f \circ g)' = (f' \circ g) \cdot g'$

2. Leibniz Notation for Single Variable

If $y = f(u)$ and $u = g(x)$, the rule is written as: $\dfrac{dy}{dx} = \dfrac{dy}{du} \cdot \dfrac{du}{dx}$

With evaluation points explicitly stated: $\left.\dfrac{dy}{dx}\right|_{x=c} = \left.\dfrac{dy}{du}\right|_{u=g(c)} \cdot \left.\dfrac{du}{dx}\right|_{x=c}$

3. Chain Rule for $n$ Functions

Given $n$ functions $f_1, \ldots, f_n$ forming a composite function, if each function $f_i$ is differentiable at its immediate input, the derivative in Leibniz's notation is: $\dfrac{df_1}{dx} = \dfrac{df_1}{df_2} \dfrac{df_2}{df_3} \cdots \dfrac{df_{n}}{dx}$

---

🛠️ Applications & Advanced Techniques

1. Composites of More Than Two Functions

To differentiate a composite of more than two functions, apply the chain rule recursively. For example, consider: $y = e^{\sin(x^2)}$

This decomposes into three functions:

• $y = f(u) = e^u \implies \dfrac{dy}{du} = e^u$
• $u = g(v) = \sin v \implies \dfrac{du}{dv} = \cos v$
• $v = h(x) = x^2 \implies \dfrac{dv}{dx} = 2x$

Applying the chain rule: $\dfrac{dy}{dx} = \dfrac{dy}{du} \cdot \dfrac{du}{dv} \cdot \dfrac{dv}{dx} = e^{\sin(x^2)} \cdot \cos(x^2) \cdot 2x$

For an arbitrarily long composition $f_1 \circ f_2 \circ \cdots \circ f_n$, defining f_{a..b} = f_a \circ f_a_+_1 \circ \cdots \circ f_b, the general formula is: $Df_{1..n} = \prod_{k=1}^{n} \left[Df_k \circ f_{(k+1)..n}\right]$

---

2. Deriving the Quotient Rule

The quotient rule is a direct consequence of combining the product rule and the chain rule.

1. Write $\dfrac{f(x)}{g(x)}$ as a product: $f(x) \cdot \dfrac{1}{g(x)}$.
2. Apply the product rule: $\dfrac{d}{dx}\left(\dfrac{f(x)}{g(x)}\right) = f'(x) \cdot \dfrac{1}{g(x)} + f(x) \cdot \dfrac{d}{dx}\left(\dfrac{1}{g(x)}\right)$
3. Compute the derivative of the reciprocal function $\dfrac{1}{g(x)}$ using the chain rule (reciprocal derivative is $-1/x^2$): $f'(x) \cdot \dfrac{1}{g(x)} + f(x) \cdot \left(-\dfrac{1}{g(x)^2} \cdot g'(x)\right) = \dfrac{f'(x)g(x) - f(x)g'(x)}{g(x)^2}$

---

3. Derivatives of Inverse Functions

Suppose $y = g(x)$ has an inverse function $f$ such that $x = f(y)$, satisfying: $f(g(x)) = x$

Differentiating both sides with respect to $x$ using the chain rule yields: $f'(g(x))g'(x) = 1$

Substituting $f(y)$ for $x$ allows us to solve for $f'$: $f'(y) = \dfrac{1}{g'(f(y))}$

Example

• Let $g(x) = e^x$, with inverse $f(y) = \ln y$.
• Since $g'(x) = e^x$: $\dfrac{d}{dy}\ln y = \dfrac{1}{e^{\ln y}} = \dfrac{1}{y}$

Warning: This formula fails if either function is non-differentiable at the evaluation point (e.g., $g(x) = x^3$ at zero, where its inverse $f(y) = y^{1/3}$ is not differentiable at zero, resulting in division by zero).

---

4. Backpropagation in Artificial Intelligence

The chain rule forms the mathematical foundation of the backpropagation algorithm, which is used in gradient descent optimization for neural networks in deep learning.

---

📈 Higher Derivatives

Faà di Bruno's formula generalizes the chain rule to higher-order derivatives. For $y = f(u)$ and $u = g(x)$, the initial derivatives follow structured compositional expansions:

• First derivative: $\dfrac{dy}{dx} = \dfrac{dy}{du} \cdot \dfrac{du}{dx}$
• Higher-order derivatives account for both powers and products of the inner and outer function derivatives.