Ask Russ Rhinehart - How does one Test Dynamic Models?

Russ' Response:

Test means to review the model predictions (output values) with both data and logical expectations.  If your model outputs adequately match the process data and expectations over a wide range of operating conditions, then your model is probably good for its intended purpose.

A problem is that the process data is infected by many uncontrolled influences.  These include measurement noise, calibration error, data discretization, and the effects of uncontrolled and unknowable disturbances.  So, even if the model were perfect, the model would not exactly fit the data. 

The other problem is that your model is never perfect.  So, even if Nature revealed the truth, the model would not perfectly match the data. 

You will not be able to claim that your model is true, or correct.  To establish credibility within the humans in your organization, you might report, “The model is fully functional.” 

Testing means that the model is sufficiently accurate for its intended purpose.  Test for sufficiency, not perfection.  Know what model attributes would make it fit for its intended purpose, and test for that.

Use both logic-based and data-based tests.  Logic-based evaluation is a set of sanity checks to explore the question, “Does the model make sense with what we know or expect?”  This validity testing decision is based on qualitative reasoning.  Data-based validation compares the model to data. 

An extended version of this pod with details and examples can be accessed under the MODELING/SIMULATION Menu on my web site, www.r3eda.com. For more information on this topic email Russ Rhinehart at [email protected].