Maxima, minima and gradient descent

Many problems ask for the best value: the most area, the least cost, the smallest error. At a smooth peak or valley the curve is flat, so the derivative is zero.

The fence problem

You have 40 m of fence for a rectangular field. If one side is x, the other is 20 − x, and the area is A(x) = x(20 − x) = 20x − x².

• A′(x) = 20 − 2x.
• A′(x) = 0 when x = 10.
• So the best field is a 10 × 10 square, with area 100 m².

Peak or valley?

The second derivative f″ tells you how the slope changes. If f″ is positive where f′ = 0, the curve bends up: a minimum. If f″ is negative, it bends down: a maximum. Here A″(x) = −2, so x = 10 is a maximum.

Gradient descent

When the equation f′(x) = 0 is too hard to solve, walk downhill instead: repeat x ← x − rate × f′(x). Each step moves against the slope. The example finds the minimum of f(x) = (x − 3)² + 1, whose slope is 2(x − 3).

def area(x):
    return x * (20 - x)

best = max(range(0, 21), key=area)
print("checking every whole side: best side", best, "area", area(best))

def slope(x):
    return 2 * (x - 3)

x = 0.0
for step in range(8):
    x = x - 0.25 * slope(x)
print("gradient descent finds x ≈", round(x, 4), "(the minimum is at 3)")

This one line is how machine-learning models are trained, with millions of variables instead of one (see the Machine Learning course).

Common mistakes

• Assuming f′ = 0 always means a maximum. It may be a minimum, or a flat point that is neither (like x³ at 0).
• A learning rate that is too big: the steps jump over the valley and can grow without end.

Mark lesson complete