Assessment of Fit

Because this is a temporal model and is intended to be used to predict future cheetah viability, some assessment of its prediction skill is of interest. The statistic used here is the root mean squared prediction error (RMSPE). The RMSPE is the square root of the mean of squared prediction errors for one time step ahead. For each prediction, the model is fitted with consistency analysis using all data that is strictly earlier than the current prediction time. Note that this is not cross-validation but true average one-step ahead prediction error.

Computing RMSPE's for individual chance nodes in the influence diagram allows prediction skill on particular nodes to be assessed and gives the RMSPE interpretable units. When the sample contains unknown errors however, the RMSPE should not be viewed as the ultimate criterion for model selection. Rather, it should be examined in light of known weaknesses in the sample. An example of such an assessment follows.

Separate RMSPE values were computed for herbivore counts and detection fraction predictions over the observation times. For c_H = .5, these calculations yielded 3757.5, and .727 for B_t and D_t, respectively. For c_H = 0., these values are 3716.9, and .720. Figure 4 gives the predicted and observed values on these two variables as a function of time for c_H = .5 and c_H = 0. The two sets of predictions are indistinguishable.

***** [Figure 4 about here] *****