An interactive Three.js laboratory demonstrating horizontal slicing, top-view level curve projection, the golden rules of contour spacing, and why gradient descent cuts contours at $90^\circ$.
In Deep Learning, even the simplest neural network—such as a single sigmoid neuron fitting two training data points $(x_1, y_1)$ and $(x_2, y_2)$—defines an error loss that depends on two parameters: the weight $w$ and the bias $b$:
$$\mathcal{L}(w, b) = \frac{1}{2}\sum_{i=1}^{2} \Big(\sigma(w x_i + b) - y_i\Big)^2$$Plotting $w$ on the $X$-axis, $b$ on the $Y$-axis, and the loss $\mathcal{L}(w, b)$ on the vertical $Z$-axis creates a 3D surface in $\mathbb{R}^3$. While 3D surfaces appear visually stunning, Professor Mitesh Khapra highlights in Module 5.3 (Slide 35) that:
Why is 3D visualization so cumbersome for optimization?
Use this interactive Three.js laboratory to inspect the complete 3D $\to$ 2D contour slicing pipeline. Rotate, zoom, drag the cutting plane, toggle projection rays, or watch Gradient Descent roll down the surface!
How does mathematics convert a 3-dimensional landscape into a flat 2-dimensional contour map? In Module 5.3 (Slide 36), Professor Khapra outlines the two-step slicing procedure:
Imagine passing a series of perfectly horizontal, flat cutting planes parallel to the parameter floor at regular intervals $\Delta c$ along the vertical loss axis:
$$\mathcal{P}_k = \Big\{(w_1, w_2, z) \in \mathbb{R}^3 \;\Big|\; z = c_k = c_0 + k\Delta c\Big\}$$Where the cutting plane intersects the 3D surface $\mathcal{S} = \{(w_1, w_2, \mathcal{L}(w_1, w_2))\}$, it cuts out a 3D intersection curve called an isoline (level curve):
$$\mathcal{C}_k = \Big\{(w_1, w_2) \;\Big|\; \mathcal{L}(w_1, w_2) = c_k\Big\}$$Every single point lying along curve $\mathcal{C}_k$ has the exact same loss value $c_k$.
Now, stand directly above the surface and look straight down along the loss axis (orthogonal projection $\pi: (w_1, w_2, z) \mapsto (w_1, w_2)$):
Each 3D slicing curve $\mathcal{C}_k$ is projected vertically down onto the 2D parameter plane $(w_1, w_2)$. In the Three.js visualizer above, toggle the "Drop Rays" button to see the vertical projection lines dropping from the 3D cut right onto the base floor!
Click "Top 2D View": the 3D surface seamlessly flattens into the familiar topographic contour map used throughout IIT Madras lectures and exams.
The single most important takeaway in Module 5.3 is learning how to read the rate of change (steepness) of a function directly from the spacing between adjacent contour curves.
| Contour Line Spacing | Physical Meaning | Gradient Magnitude $\|\nabla L\|$ | Gradient Descent Behavior |
|---|---|---|---|
| Small distance (Tightly packed) | Steep slope in that direction | Large $\|\nabla L\|$ | Big update steps ($w_{t+1} = w_t - \eta \nabla L$). Fast descent! |
| Large distance (Widely spaced) | Gentle slope in that direction | Tiny $\|\nabla L\|$ | Microscopic update steps. Optimization crawls! |
| Vast empty gap (Plateau) | Flat plateau ($\|\nabla L\| \approx 0$) | Near zero | GD virtually freezes; requires hundreds of epochs to cross. |
Consider two adjacent contour lines separated by a constant loss difference $\Delta \mathcal{L} = c_{k+1} - c_k = \text{const}$. Let $\Delta d$ be the perpendicular spatial distance between the two lines on the $(w_1, w_2)$ plane.
By first-order Taylor expansion:
$$\Delta \mathcal{L} \approx \|\nabla \mathcal{L}\| \cdot \Delta d \implies \|\nabla \mathcal{L}\| \approx \frac{\Delta \mathcal{L}}{\Delta d}$$Because $\Delta \mathcal{L}$ is held constant by our regular slicing intervals:
$$\text{Distance between contours } \Delta d \propto \frac{1}{\|\nabla \mathcal{L}\|}$$Therefore, wherever the gradient is huge (steep wall), $\Delta d$ must be tiny (tightly crammed contours). Wherever the gradient is tiny (gentle plain), $\Delta d$ must be huge (sparse contours). That is why reading contour density gives you an immediate physical intuition of the gradient vector field!
In Week 4 notes and live sessions, a key question repeatedly tested in PYQs is: "In what direction does vanilla Gradient Descent move on a contour plot?"
Let $\mathbf{r}(t) = \big(w_1(t), w_2(t)\big)$ be a parameterized contour line on which the loss is constant: $\mathcal{L}(\mathbf{r}(t)) = C$.
Differentiating both sides with respect to $t$ using the multivariable chain rule:
$$\frac{d}{dt}\mathcal{L}(\mathbf{r}(t)) = \frac{\partial \mathcal{L}}{\partial w_1} \frac{dw_1}{dt} + \frac{\partial \mathcal{L}}{\partial w_2} \frac{dw_2}{dt} = \nabla \mathcal{L}(\mathbf{r}(t)) \cdot \mathbf{r}'(t) = 0$$Here, $\mathbf{r}'(t)$ is the tangent vector pointing along the contour curve. Because their dot product is zero:
$$\nabla \mathcal{L} \perp \mathbf{r}'(t) \quad \text{and} \quad -\nabla \mathcal{L} \perp \mathbf{r}'(t)$$Conclusion: The gradient vector $\nabla \mathcal{L}$ is strictly normal (orthogonal) to the contour curve at every single point! Since vanilla gradient descent steps along $-\eta \nabla \mathcal{L}$, every vanilla GD step begins perpendicular to the contour line passing through the current point.
In the Three.js visualizer above, click any point on the surface or base map. Notice how the green arrow ($-\nabla \mathcal{L}$) always forms a perfect $90.0^\circ$ right angle with the yellow tangent line along the contour!
"The figure below shows the contours of a 3D surface. Some points in the figure are marked with labels (a, b, c, d, e) that correspond to either maxima or minima. Can you determine whether the labels correspond to minima or maxima without color coding?"
Correct Answer: It is NOT possible to decide whether labels correspond to minima or maxima without color coding or numerical height labels on the map.
Why is this the case?
In Modules 5.4 and 5.5, the course introduces Momentum and NAG specifically because vanilla GD struggles when contours form elongated, anisotropic valleys (like $0.1 w_1^2 + w_2^2$). Select surface 1 in our visualizer and test all three optimizers:
Update rule: $\mathbf{w}_{t+1} = \mathbf{w}_t - \eta \nabla \mathcal{L}(\mathbf{w}_t)$
Update rule: $\mathbf{u}_t = \beta \mathbf{u}_{t-1} + \nabla \mathbf{w}_t; \; \mathbf{w}_{t+1} = \mathbf{w}_t - \eta \mathbf{u}_t$
Update rule: $\mathbf{u}_t = \beta \mathbf{u}_{t-1} + \nabla\mathcal{L}(\mathbf{w}_t - \beta \mathbf{u}_{t-1}); \; \mathbf{w}_{t+1} = \mathbf{w}_t - \eta \mathbf{u}_t$
Rather than evaluating the gradient at the current position $\mathbf{w}_t$, NAG computes the gradient at the lookahead position $\mathbf{w}_t - \beta \mathbf{u}_{t-1}$. If the momentum is about to carry the particle up the steep opposing contour wall, the lookahead gradient detects the climb in advance and applies braking, drastically damping oscillations!
Question: Consider the contour lines of $f(x, y) = 0.1x^2 + y^2$. Three different gradient descent runs are shown with learning rates $\eta_1, \eta_2, \eta_3$. One run converges monotonically along $y$, one lands on $y=0$ in exactly 1 step, and one oscillates across the $x$-axis. Determine the ordering and the exact value of $\eta_3$.
Solution:
The update rule for coordinate $y$ is:
$$y_{t+1} = y_t - \eta \frac{\partial f}{\partial y} = y_t - \eta(2y_t) = (1 - 2\eta)y_t$$Hence: $\mathbf{\eta_2 < \eta_3 < \eta_1}$, and $\mathbf{\eta_3 = 0.5}$.
Question: "Contours are tall ellipses where contour lines are closer along the $x$-axis than along the $y$-axis. A person walks from $-1$ to $+1$ on both axes. Does the slope change more rapidly along $x$ than along $y$?"
Solution: True. Closer contour lines directly signify a steeper slope and a faster rate of elevation change per unit distance traveled.
Question: Under vanilla GD with fixed learning rate $\eta$, select all true statements about step sizes:
Solution: The step size is $\Delta w = \eta \|\nabla \mathcal{L}\|$. Because $\|\nabla \mathcal{L}\|$ is large on steep surfaces and tiny on gentle surfaces, the update is large on steep regions (crammed contours) and tiny on gentle regions (sparse contours).