Author(s): Samit Ganguly

Originally published on Towards AI.

Most of the space your model works in is empty. Knowing where the data actually sits fixes all real bugs.

Imagine yourself as an ML engineer, and your diffusion model has been producing nothing but grey, washed-out mush, which you cannot debug.

The article uses high-dimensional geometry to explain why “simple” operations in latent space (like averaging two vectors) can push models into regions they were never trained on. It introduces an analogy of an orange to show that in many dimensions, almost all volume lies in a thin “peel” near the boundary and the interior is effectively empty, so straight-line interpolation (Lerp) can cut through an unseen middle while spherical interpolation (Slerp) stays on the data shell. From there, it connects the same principle to nearest-neighbor distance concentration, the behavior of Gaussian samples in high dimensions (the Gaussian annulus), and common ML scaling and similarity choices: embeddings should be interpreted by direction (cosine similarity) rather than proximity to the origin, and transformer attention uses √d normalization to correct geometry-induced magnitude growth. The takeaway is that embeddings and latents live on a shell, so debugging should focus on whether your computations respect that geometry.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI