Author(s): Maedeh Torkian

Originally published on Towards AI.

Learning to Say “I Don’t Know”: The First Floor of Awareness

Essay #4 in the Humble Model Series

I. Introduction: The Flicker Becomes a Voice

In the bridge essay, we named the pattern we had seen across three experiments:

– Essay #2: A model, confident near 1.0000, could not distinguish the robust majority from the fragile minority. It was silent about its own limits.

The Attack Landscape: Why Silence Is the Vulnerability

Essay #2 in the Humble Model Series

pub.towardsai.net

– Essay #3: We built walls — FGSM training, PGD training, defensive distillation. Each raised the bar. Each eventually showed cracks. Robustness training fixed 17 of 18 images, but the one that remained wrong still received confidence 0.6187 — a flicker of hesitation that never became a warning.

Walls, Shields, and Illusions: Defenses and Their Limits

Essay #3 in the Humble Model Series

pub.towardsai.net

– The Bridge: We turned from resistance to awareness. We asked: What if the goal is not to be unbreakable, but to be undeluded? We said awareness needs two things — calibration and a gate — and that Essay #4 would build both.

The Bridge: From Resistance to Awareness

Why Defenses Alone Will Never Be Enough

medium.com

This is that essay. We build the voice.

We will move from the flicker (0.6187) to a calibrated uncertainty estimate — a number that actually means something, a number that can be acted upon. But it is worth being precise about what kind of essay this is. Uncertainty quantification is not the end of the series’ argument; it is the first floor of an awareness architecture. A model that can say “I am not sure” has not yet said “this input is strange” or “my decision boundary is fragile here” — those are separate rooms in the same building, and later essays will build them. This essay’s job is narrower and more foundational: give the model a voice at all.

We will implement three practical mechanisms that make that voice possible:

1. Monte Carlo Dropout — using stochastic inference to estimate uncertainty.

2. Deep Ensembles — training multiple models and measuring their disagreement.

3. Out-of-Distribution Detection — recognizing when an input is unlike anything the model has seen.

These are means, not the point. The point is what we build with them: a decision gate — a simple, auditable rule that says: “If uncertainty is high, do not predict. Ask for help.”

This is not a new wall. It is a voice, and it is the first floor of a structure this series will keep building on.

II. What Is Uncertainty — and Why Softmax Is Not Enough

Awareness is not binary. A model can be aware of its uncertainty about its own prediction (epistemic self-knowledge). It can be aware that the input in front of it looks nothing like its training data (distributional awareness). It can be aware that its decision boundary near this particular input is fragile, even when its own confidence doesn’t show it (geometric awareness — the flicker at 0.6187 from Essay #3 was exactly this, unreported). This essay builds the first of those three: the model’s awareness of its own uncertainty. Out-of-distribution detection, which we also implement here, begins to reach toward the second. The decision gate is what connects either signal to an action. Later essays in this series can push further into geometric and deployment-level awareness; this one lays the floor they will stand on.

Most deep learning models output a softmax vector — a probability distribution over classes. It is tempting to read this as uncertainty. But it is not.

A softmax probability is:

• Conditional on the input and the training distribution.

• Not calibrated to actual accuracy rates.

• Silent about whether the input is out-of-distribution, adversarial, or simply ambiguous.

A model that outputs 0.99 on a clean image and 0.99 on an adversarial image is not expressing uncertainty. It is expressing confidence — whether earned or not.

Uncertainty quantification aims to close this gap. It gives the model a way to say:

– “I am not sure about this prediction.”

– “This input is unlike anything I have seen.”

– “You should not trust this output.”

There are two types of uncertainty:

Aleatoric uncertainty cannot be reduced with more data. Epistemic uncertainty can — but only if the model has a way to detect when it is out of its depth.

III. Method 1: Monte Carlo Dropout

Dropout is a regularization technique: during training, random neurons are “dropped” (set to zero) to prevent over-reliance on specific features.

Monte Carlo (MC) Dropout uses dropout during inference. Instead of a single forward pass, we run the model multiple times with dropout enabled. The predictions vary — and the variance gives us an estimate of epistemic uncertainty.

How It Works

1. Train a model with dropout (standard practice).

2. At inference, enable dropout and run N forward passes (e.g., 50–100).

3. Compute:

• Mean prediction — the average softmax vector.

• Variance — how much the predictions vary across passes.

Trade-offs

Code Snippet (PyTorch)

def mc_dropout_predict(model, image, num_passes=50):
 model.train() # Enable dropout
 predictions = []
 for _ in range(num_passes):
 with torch.no_grad():
 pred = torch.softmax(model(image), dim=1)
 predictions.append(pred)
 model.eval() # Disable dropout for normal use
 predictions = torch.stack(predictions) # [N, 1, 10]
 mean_pred = predictions.mean(dim=0)
 variance = predictions.var(dim=0)
 return mean_pred, variancepy

Results :

Adversarial-Notebooks/#4/Notebook_#4.ipynb at master · Maee127/Adversarial-Notebooks

A series of articles to more study in Adversarial attacks, defenses and train …

github.com

On a clean, easily-classified MNIST test digit (the digit 7 at test index 0), MC Dropout with 50 passes returned confidence 1.0000 and variance 0.0000 — the model is not just confident, it is unanimous across every stochastic pass. That single example is a fine sanity check but not evidence on its own; the aggregate numbers over 200 images per condition are the ones worth trusting:

Variance rises about 31× from clean to OOD, and only about 6× from clean to adversarial. That six-to-thirty-one ratio is the first quantitative sign of the gap this essay is ultimately about: MC Dropout notices unfamiliarity far more than it notices adversarial perturbation, even before the decision gate does anything with the number.

IV. Method 2: Deep Ensembles

Deep Ensembles are a simpler — and often more effective — alternative to MC Dropout. The idea: train multiple models with different initializations, and measure their disagreement.

How It Works

1. Train M models (e.g., 5) on the same dataset, but with different random seeds.

2. At inference, run all M models on the input.

3. Compute:

• Mean prediction — the average of the M softmax outputs.

• Disagreement — the variance across the M predictions.

Trade-offs

Code Snippet (PyTorch)

def ensemble_predict(models, image):
 predictions = []
 for model in models:
 model.eval()
 with torch.no_grad():
 pred = torch.softmax(model(image), dim=1)
 predictions.append(pred)
 predictions = torch.stack(predictions) # [M, 1, 10]
 mean_pred = predictions.mean(dim=0)
 variance = predictions.var(dim=0)
 return mean_pred, variancep

Results :

Adversarial-Notebooks/#4/Notebook_#4.ipynb at master · Maee127/Adversarial-Notebooks

A series of articles to more study in Adversarial attacks, defenses and train …

github.com

On the same clean test digit, the 5-member ensemble agrees with MC Dropout’s read: confidence 1.0000, variance 0.0000. The aggregate numbers tell a slightly different story from MC Dropout’s, though:

The ensemble’s variance rises about 28× from clean to OOD and about 4× from clean to adversarial — the same lopsided pattern as MC Dropout, at a slightly different scale. Both methods agree on the shape of the problem even though they disagree on the exact numbers: distributional unfamiliarity moves the needle far more than adversarial perturbation does.

V. Method 3: Out-of-Distribution Detection

Uncertainty is not just about confidence variance. Sometimes, the input is simply strange — unlike anything the model has seen during training.

OOD detection asks: Is this input from the same distribution as the training data?

How It Works (Simple Approach)

1. Train a model on the training set.

2. At inference, compute a distance metric between the input and the training distribution:

• Distance in feature space — how far is the input from the nearest training example in the model’s latent space?

• Reconstruction error — if using an autoencoder, can the input be reconstructed well?

• Density estimation — how probable is the input under a density model trained on the training data?

We implement the simplest approach: minimum distance in feature space, against a cached bank of 1,000 training features.

Code Snippet (PyTorch)

def extract_features(model, dataloader, max_samples=1000):
 features = []
 model.eval()
 with torch.no_grad():
 for images, _ in dataloader:
 _, feat = model.forward_with_features(images)
 features.append(feat.cpu())
 if len(features) * images.size(0) >= max_samples:
 break
 return torch.cat(features, dim=0)[:max_samples]

def compute_ood_score(feature, training_features):
 # Minimum distance to any training feature
 return torch.cdist(feature.unsqueeze(0), training_features).min().item()

Results:

Adversarial-Notebooks/#4/Notebook_#4.ipynb at master · Maee127/Adversarial-Notebooks

A series of articles to more study in Adversarial attacks, defenses and train …

github.com

This section went through three distinct versions before landing on something trustworthy, and the path there is worth showing rather than hiding, because each wrong turn taught something the final number doesn’t convey on its own.

First attempt — a single pair of examples. A clean MNIST test digit sat at distance 8.3552 from its nearest training-feature neighbor; a Fashion-MNIST image sat at 23.7167 — nearly three times farther. That looked like a clean signal. It wasn’t evidence of anything, because it was one pair of numbers.

Second attempt — measured properly, and it failed. Cached as a Mahalanobis distance (normalizing by the training features’ covariance) and measured over 500 images per condition instead of one:

Cohen’s d between the two: 0.00. No separation. The single-pair comparison above was two draws from almost entirely overlapping distributions that happened to land on opposite sides of their means. The cause: this Mahalanobis distance was fit as one global Gaussian over all ten digit classes’ features pooled together. MNIST’s classes occupy genuinely different regions of a 128-dimensional feature space — the true structure is multimodal — and a single covariance estimate averages over that structure instead of respecting it.

Third attempt — class-conditional, and it worked. Fitting one Gaussian per digit class and scoring against the nearest one (Lee et al., 2018), measured again over 500 images per condition:

Cohen’s d: 0.99 — a large effect, and this time built from a population, not a pair. Class-conditional Mahalanobis distance genuinely separates MNIST digits from Fashion-MNIST images. The lesson from the failed global version generalizes past this one metric: a feature space shaped by ten distinct classes will defeat any detector that assumes it’s shaped like one blob, however sensible that detector looks on paper.

A second bug, found only once fusion was attempted. Validating the metric wasn’t the end of it. The first attempt to use this class-conditional score inside the decision gate — at threshold 18.29, the midpoint of the two means above — still deferred on 100% of both clean and OOD inputs. The metric was sound; the code applying it wasn’t. The gate was scoring each image’s features from an ensemble member’s forward pass, while the Mahalanobis Gaussians had been fit on a different model’s (the baseline’s) feature space. Two independently-initialized networks, even with identical architecture, don’t share an aligned feature space — there’s no reason hidden dimension 37 means the same thing in both. Distances computed that way are noise dressed up as a number. Section VI reports what happened once that mismatch was fixed and the score was computed consistently against the model it was actually fit on.

VI. The Decision Gate

Uncertainty becomes useful only when it acts. The decision gate is the mechanism that translates uncertainty into action.

A Simple Rule

If uncertainty > threshold, do not predict. Defer to a human or a fallback system.

We use two thresholds:

Implementation

def decision_gate(models_or_model, image, method='ensemble',
 threshold_variance=0.05, threshold_confidence=0.7,
 num_passes=50):
 if method == 'mc_dropout':
 mean_pred, variance = mc_dropout_predict(models_or_model, image, num_passes)
 elif method == 'ensemble':
 mean_pred, variance = ensemble_predict(models_or_model, image)
 else:
 raise ValueError("method must be 'mc_dropout' or 'ensemble'")

 confidence = mean_pred.max().item()
 uncertainty = variance.max().item()

 if uncertainty > threshold_variance or confidence < threshold_confidence:
 return 'DEFER', None, uncertainty, confidence
 else:
 return 'PREDICT', mean_pred.argmax().item(), uncertainty, confidence

A First Look: Two Single Images

On the clean MNIST digit: PREDICT, confidence 1.0000, uncertainty 0.0000 — the gate lets it through, correctly.

On a Fashion-MNIST image: DEFER, confidence 0.5254, uncertainty 0.1664 — the gate catches it, correctly, using variance and confidence alone, with no OOD distance signal involved at all. That the gate defers on this OOD input even without the feature-distance check is itself informative: the ensemble’s disagreement on an unfamiliar input is doing real work, independent of the separate distance metric in Section V.

Evaluation Metrics

The goal is to balance coverage and risk: defer when uncertain, but not so often that the system becomes unusable.

Full Evaluation — 10,000 Images per Set

Read together, these two rows are the empirical heart of this essay. On clean MNIST, the gate defers only 2.52% of the time — and when it does predict, it is right 99.85% of the time, actually higher than the 99.00% baseline accuracy, because the gate is filtering out exactly the inputs the ensemble finds shakiest. On Fashion-MNIST, the gate defers 80.09% of the time. That is the awareness working as intended: four out of five OOD images never reach a prediction at all.

But the remaining 19.91% it does predict on is where the honesty has to continue. Of those, only 5.68% are “correct” against MNIST labels that do not meaningfully apply to a shirt or a shoe — the accuracy number is close to meaningless here by construction, and the risk figure of 94.32% is the number that matters: when the gate lets an OOD input through, it is wrong almost every time. Confidence-and-variance thresholds catch most distributional shift, but not all of it, and the failures that slip through are close to total failures, not partial ones. This is precisely the gap the OOD distance metric in Section V was built to close.

The fused gate, once the metric and the code both worked. With the class-conditional OOD score correctly wired in (threshold 18.29, scored consistently against the model it was fit on):

This is not the clean win the notebook’s early single-example numbers suggested, and it is not the total failure the two broken runs before it produced either. It’s a real, mixed result, and it deserves to be read carefully rather than rounded off in either direction. Fusion pushes coverage down substantially on both datasets — clean coverage drops from 97.48% to 87.04%, a real cost, not a rounding error. On OOD, coverage drops further still, from 19.91% to 7.62%. But look at the risk column: on OOD, risk-among-predictions barely moves, 94.32% to 95.01% — fusion is not making the gate any better at picking out which OOD inputs are safe to predict on. What it’s doing is shrinking the number of OOD inputs it’s willing to predict on at all. In absolute terms, on 10,000 Fashion-MNIST images, the unfused gate produces roughly 1,878 silent wrong predictions; the fused gate produces roughly 724 — a 61% reduction in the raw count of confidently-wrong OOD outputs. That’s a genuine improvement in the failure mode this whole essay is about. It comes paid for by clean data: the fused gate now defers on real, easy, correctly-classifiable MNIST digits about five times more often than the unfused one did, cutting the system’s usable coverage by ten percentage points to prevent OOD failures it still, on a per-prediction basis, doesn’t understand any better than before.

A third condition — adversarially perturbed MNIST, not shown in this table — turns out to matter more than either of the two above. Section VII reports it in full, but the short version: the same gate that defers on 80.09% of Fashion-MNIST images defers on only 8.80% of FGSM-perturbed ones. Distributional awareness and adversarial awareness are not the same skill, and this gate has only really demonstrated the first. It’s also worth noting that ε=0.25 is not a cherry-picked weak attack: Essay #2’s minimum-epsilon search on this same model found a median flip-epsilon of 0.24 across successfully-attacked images, so ε=0.25 sits right at the typical strength needed to fool this baseline at all — the low deferral rate is not an artifact of testing with too gentle a perturbation.

VII. Putting It All Together: The Humble Model — Awareness, Not Yet Complete

We now have three components:

1. Uncertainty Quantification (MC Dropout or Deep Ensemble)

2. OOD Detection (distance in feature space)

3. Decision Gate (threshold-based deferral)

The humble model is not a single method. It is meant to be a system that integrates these components — an awareness architecture: epistemic measurement (the ensemble) telling the gate when the model itself is unsure, distributional detection (feature distance) telling it when the input itself is unfamiliar, and a policy layer (the gate) turning either signal into a decision to predict or defer, rather than treating confidence as a fixed, unquestioned output.

The Flow — As Designed

1. Input arrives.

2. OOD detector checks if the input is strange. If yes → defer.

3. Uncertainty estimator runs (MC Dropout or ensemble) to produce a distribution.

4. Decision gate checks uncertainty and confidence. If either exceeds threshold → defer.

5. If not deferred → predict with confidence estimate.

The Flow — As Actually Implemented, and What It Took to Fuse It Correctly

The notebook builds and demonstrates all three components, but the full evaluation in Part I only exercises step 3 and step 4 by default — the OOD distance score from Section V is computed separately from decision_gate unless explicitly fused in. What the unfused rows of Table VI measure is how far ensemble disagreement alone can carry an awareness system on natural distribution shift: quite far (97.48% coverage at 0.15% risk on clean data, 80.09% deferral on OOD data) but not all the way (94.32% risk on the fraction of OOD inputs that still slip through).

Getting the fused version right took three attempts, and each failure was informative in its own way. The first fusion attempt, at a threshold derived from a global Mahalanobis distance that turned out not to separate the two distributions at all (Section V), deferred on 100% of both clean and OOD inputs — not a tuning problem, but a genuinely non-separating metric being asked to draw a line that didn’t exist. The second attempt, after switching to a class-conditional Mahalanobis distance that did separate the distributions with a large effect size (Cohen’s d = 0.99), still deferred on 100% of both — this time because the gate was scoring one model’s features against Gaussians fit on a different model’s feature space, a mismatch invisible in the code until the symptom (universal deferral, again) forced a closer look. Only the third attempt, scoring consistently against the same model the metric was fit on, produced the numbers in Table VI’s fused rows.

The lesson that generalizes past this one experiment: an “any of these signals trips the alarm” gate is only as trustworthy as its least-validated input, and a signal can fail for two entirely different reasons — being genuinely uninformative, or being informative but wired in wrong — that look identical from the outside (100% deferral, both times) and require different fixes. Measuring at the population level caught the first kind of failure. It took a suspicious result and a code review to catch the second.

What the working fused gate actually buys, once both bugs were fixed, is reported in Section VI: a real reduction in absolute OOD failures, bought at a real cost to clean-data coverage, without making the gate any more discerning about which OOD inputs are safe to trust. That is a more honest resolution than either “fusion breaks everything” or “fusion solves everything,” and it’s the one the data actually supports.

Adversarial Inputs — Still the Open Wound

The original plan for this notebook included evaluating the humble model on adversarial MNIST, generated the same way as in Essays #2 and #3. That run is in the notebook (Part K, FGSM at ε=0.25, unfused gate), and it remains the most important number in this essay — nothing about the OOD-fusion resolution above changes it, because fusion was never tested against the adversarial condition:

The gate defers on natural distribution shift nine times more often than it defers on adversarial perturbation of the same underlying digits. This is not a bug in the gate — it is exactly what the design predicts, once stated plainly: ensemble disagreement measures how unfamiliar an input looks to independently-trained models, and a well-constructed FGSM perturbation is deliberately small enough, and aligned enough with the model’s own gradient, to leave that sense of familiarity almost untouched. Essay #2’s own minimum-epsilon search found a median flip-epsilon of 0.24 across successful attacks, so ε=0.25 was a representative attack strength, not an artificially gentle one chosen to flatter the result. The finding that matters is the deferral gap, not the risk number: this awareness architecture, as built, is watching for the wrong kind of strangeness. It notices a shirt, eventually, after three attempts at getting the detector right. It does not reliably notice a 9 that has been nudged toward an 8, and nothing in this essay has yet tested whether the class-conditional Mahalanobis distance that rescued the OOD story would do any better against adversarial inputs than the ensemble variance already does. That is the next open question, not a settled one.

VIII. What We Now Know — and What We Still Do Not

• Answered, and not the comfortable answer: ensemble-disagreement-based uncertainty does not reliably distinguish adversarial examples the way it distinguishes natural distribution shift. Deferral on FGSM-perturbed inputs (8.80%) is roughly a ninth of deferral on Fashion-MNIST (80.09%), even though both are, in a loose sense, “inputs the model wasn’t trained on.” Awareness of unfamiliarity and awareness of adversarial manipulation are different competencies, and this essay’s gate has only demonstrated the first.

• Answered, finally, with a real number — and it’s neither the win nor the failure it looked like along the way: fusing OOD detection into the decision gate works, once the detector is class-conditional and scored against the right model’s features, but “works” means something specific and limited. On Fashion-MNIST, fusion cuts absolute silent failures by roughly 61% (about 1,878 wrong predictions down to about 724, out of 10,000 images) by predicting on far fewer OOD inputs overall — but it does this without getting any better at telling which OOD inputs are safe (risk-among-predictions moves from 94.32% to 95.01%, essentially unchanged). It buys that reduction by deferring on clean MNIST roughly five times more often too, cutting usable coverage from 97.48% to 87.04%. Two earlier attempts at this same fusion returned 100% deferral on everything — once because the underlying metric didn’t separate the distributions at all, once because the code scored the wrong model’s features against it. Both looked identical from the outside and required different fixes.

• Corroborating detail: Essay #2’s own minimum-epsilon search found a median flip-epsilon of 0.24 across successful attacks (out of 500 images, only 22 flipped within ε≤0.40 at all). The ε=0.25 used for this essay’s FGSM test sits almost exactly on that median — this was a representative attack strength for this model, not an unusually weak one chosen to make the gate look better than it is.

• Still open: whether the class-conditional Mahalanobis distance that finally separated clean MNIST from Fashion-MNIST also separates clean from adversarial inputs — it was never tested against the FGSM condition. If it does, fusion might close some of the adversarial deferral gap above in a way ensemble variance alone cannot; if it doesn’t, that would be a second piece of evidence that adversarial perturbations are specifically built to evade exactly this kind of feature-space detection, not just ensemble disagreement.

• Still open: whether a stronger or more varied adversarial attack budget (this test used a single ε=0.25 FGSM pass against an undefended baseline) changes the 8.80% deferral figure, and whether an ensemble of defensively trained models (PGD training, from Essay #3) would show more disagreement on adversarial inputs than this undefended one does.

• How to set thresholds without a labeled validation set of uncertain inputs.

• Whether the decision gate works in high-stakes, low-latency environments where deferral is not possible.

. Whether the humble model is truly humble, or simply more sophisticated in its silence — on natural data, it earns that description; on adversarial data, not yet.

IX. Invitation

You have seen the silence. You have built the walls. You have crossed the bridge.

Now you have the first floor of a voice.

Not a wall. A boat.

Not resistance. Awareness.

Not certainty. Honesty.

The humble model is within reach — one floor of it, built and measured here; the rest, named and left standing open for what comes next.

The boat is built. The voice is forming. But it does not yet speak when it matters most — against adversarial inputs. That is the next shore.

Come. Walk with me into deeper fog.

X. References

• Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv:1412.6572.

• Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems (NeurIPS), 30.

• Lee, K., Lee, K., Lee, H., & Shin, J. (2018). A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Processing Systems (NeurIPS), 31.

• Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. International Conference on Machine Learning (ICML), 1050–1059.

• Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013). Intriguing properties of neural networks. arXiv:1312.6199.

• Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083.

• Papernot, N., McDaniel, P., Wu, X., Jha, S., & Swami, A. (2016). Distillation as a defense to adversarial perturbations against deep neural networks. IEEE Symposium on Security and Privacy (S&P), 582–597.

• Carlini, N., & Wagner, D. (2016). Defensive distillation is not robust to adversarial examples. arXiv:1607.04311.

• Athalye, A., Carlini, N., & Wagner, D. (2018). Obfuscated gradients give a false sense of security. International Conference on Machine Learning (ICML), 274–283.

• Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Madry, A. (2018). Robustness may be at odds with accuracy. arXiv:1805.12152.

• Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., & Jordan, M. (2019). Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning (ICML), 7472–7482.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI