In our previous post, How to Spoil a Perfectly Good Model, we looked at the common pitfalls that trip up even experienced practitioners. Now, we’ll consider modeling from another perspective – the decision maker’s perspective – and ask how to build models they feel they can rely on, and how to convey that sense of reliability.
But to do this, we need to consider one question:
If every model is a simplification of the real world – never perfect, never “true” – then how do we judge its reliability?
First, let’s clarify what we mean by reliability. For our purposes, reliability doesn’t mean statistical reliability or a calculated margin of safety. It means something more practical: the sense that a model’s outputs can be relied upon to inform a decision. In groundwater modeling, that sense can’t be proven with a number or a validation score, because no model result is truly unique. Instead, reliability grows from how the model is built, tested, and interpreted – and how openly its limitations are shared.
However, while it’s tempting to equate reliability with certainty, certainty is rarely possible in groundwater modeling – and that’s where the language of “validation” often leads us astray.
Why “validated” doesn’t equal reliable
For decades, modeling culture has treated “validation” as the final proof of correctness – as though a model could ever perfectly mirror reality.
However, that’s a misleading idea. In practice, validation often implies finality and the notion that a model can be proven correct. In groundwater modeling, validation often takes a mathematical form: statistical checks like the root mean square error (RMSE) or correlation coefficients between measured and simulated heads. These metrics appear precise, often to several decimal places, and that precision gives the comforting illusion that the model itself must also be precise.
The problem is, a good RMSE does not mean the model is right. A good RMSE only means that the model can reproduce certain observations. For example, many different combinations of parameters, boundary conditions, and even conceptual frameworks can yield nearly identical fits to the same data. This non-unique nature of groundwater models is what makes “validation” such a slippery term – one that can easily undermine the very idea of a model’s reliability.
As D. K. Nordstrom (2012) cautioned, “model validation” is a myth. Models are simplifications – and simplifications, by definition, can never be perfect representations of the real world. Clifford Voss (1998) made a similar point, warning against the delusion of validation and the myth of the single number as measures of model success. We can, and should use validation metrics like RMSE as lines of evidence that a model behaves similarly to reality in some respects, but they are not proof that the model is reliable.
Ultimately, model reliability, then, isn’t the product of a one-time validation. It’s earned through repeated evaluation – by showing that a model behaves credibly within its intended context, and by being transparent about where it doesn’t.
Credibility Creates Reliability
If validation can’t prove a model is reliable, what can?
I’d suggest that the sense of reliability – or trust in a model – must be deliberately built. We build it by actively adding credibility as we develop the model, step by step, decision by decision. As a result, credibility isn’t a by-product; it’s something we pursue on purpose.
We can build credibility through corroboration, by comparing model predictions with real-world observations; through evaluation, by asking critical questions at every stage; and through transparency, by showing how the model was constructed, what assumptions were made, and where the uncertainties lie.
That means continually asking:
- Were the principles of groundwater science applied correctly?
- Is the data sufficient and representative of the system?
- Do the results make sense when compared with what’s observed in the field?
- Does the model actually serve the purpose it was built for?
In short, these aren’t checklist questions — they’re a mindset. They keep the focus on using the model to understand and support decisions, not just to generate numbers.
Finally, iteration is the last part of this process. A model gains credibility when it continues to make sense under new information, changing conditions, or what-if scenarios. Each test, sensitivity run, and comparison helps us learn — refining not just the model, but our understanding of the system itself.
In summary, reliability, as noted by Thompson (2022) as well as Kay and King (2020), isn’t a mathematical property; it’s a professional judgment. It’s the sense that a model has been developed with care, tested against reality, and communicated with honesty. In other words, reliability grows out of a deliberate effort to build credibility.
Final Thoughts: The Path to Reliability
Ultimately, nobody really wants a model – they want what it enables: understanding, judgment, and a way forward. The model is just a means to that end. For it to serve that purpose, decision-makers need a sense that it can be relied upon. That sense of reliability isn’t a statistic; it’s a feeling that the model behaves as expected, within the limits of its purpose.
We should use validation results and fit metrics to help build some confidence, in the model. However, we need to recognize and communicate that validation statistics are only a line of evidence. A model can score beautifully on error statistics and still be conceptually wrong.
The practical way to earn that sense of reliability is by building credibility into the model itself — showing how assumptions were made, how results compare with field observations, and how new information changes what we know.
When we approach modeling this way, reliability stops being a checkbox and becomes a relationship — between modeler, model, and decision-maker. That’s how models truly earn trust and fulfill their purpose: not by being perfect, but by helping us think more clearly about the world we’re trying to shape.
Further Reading
Barbour, S. L., & Krahn, J. (2004). Numerical modelling – prediction or process? Geotechnical News, 22(4), 44–52. Publisher link
Kay, J. A., & King, M. A. (2020). Radical Uncertainty: Decision-Making Beyond the Numbers. W. W. Norton & Company. Publisher link — ISBN 9781324050807
Nordstrom, D. K. (2012). Models, validation, and applied geochemistry: Issues in science, communication, and philosophy. Applied Geochemistry, 27(10), 1899–1919.
Starfield, A. M., Smith, K. A., & Bleloch, A. L. (1994). How to model it: Problem solving for the computer age. McGraw-Hill. ISBN 0808779702. https://dl.acm.org/doi/10.5555/562530
Tetlock, P. E., & Gardner, D. (2015). Superforecasting: The Art and Science of Prediction. Penguin Random House. ISBN 9780804136716. https://www.penguinrandomhouse.com/books/227815/superforecasting-by-philip-e-tetlock-and-dan-gardner/
Thompson, E. (2022). Escape from Model Land: How Models Can Lead Us Astray and What We Can Do About It. Basic Books. ISBN 9781541612842 https://www.hachettebookgroup.com/titles/erica-thompson/escape-from-model-land/9781541612842/
Voss, C. I. (1998). Editor’s Message: Groundwater modeling — simply powerful. Hydrogeology Journal, 6(4), A4–A6. Available via ResearchGate