I like the way the abstract of this paper distinguishes between (1) the accuracy of a model as measured by comparing it to physical observations (always assuming those are an accurate or at least unbiased measurement of the true state of the universe and (2) the appropriateness of a model to be used in decision making. I find these concepts very, very difficult to get across even to scientists and engineers.

Ecological forecasting models: Accuracy versus decisional quality

We consider here forecasting models in ecology or in agronomy, aiming at decision making based upon exceeding a quantitative threshold. We address specifically how to link the intrinsic quality of the model (its accuracy) with its decisional quality, ie its capacity to avoid false decisions and their associated costs. The accuracy of the model can be evaluated by the [Greek symbol rho – I don’t know what they mean by this just from reading the abstract] of the regression of observed values versus estimated ones or by the determination coefficient. We show that the decisional quality depends not only of this accuracy but also of the threshold retained to make the decision as well as on the state of nature. The two kinds of decisional errors consists either in deciding no action while an action is required (false negatives) or to act while it is useless (false positives). We also prove that the costs associated to those decisions depend also both of the accuracy of the model and of the value of the decision threshold.

Ecological Modeling