Adding a level or not/increasing residual variance

[thomasPFF]

Dear Forum,

In the final stages of my cross-classified analysis of pricing in the bookmarket, I am pondering the adding of an extra classification in my cross-classification model. I have editions of books which are nested in authors and publishers but as some booktitles are represented in multiple editions (e.g. paperback, hardcover) it could be said that the booktitle should be added as well.

When I add this extra level in the model some variables effects shift with 0,001 and in the full model there is a variance of 0,2 percent located on this level. In the LEMMA course the Bayesian DIC to judge whether a level should be added or not, in my case the model without the level has a DIC of 12125,51 while with the extra level it is 12092,24. The question then is, what DIC-difference dictates that one should add the extra level? Is this difference enough to demand adding it while there is little effect in the model?

The second question involves increasing residual variance and the procedure runmlwin uses. It does a normal nested multilevel first and then uses those figures for the MCMC. When I add the extra level in the empty model and in the fourth last model, there is no variance at all on this level in the normal multilevel model which estimates are stored and used in the MCMC. In the two intermediary models there is however variance a little variance also on this level. I guess due to this anomaly the residual variance in the second and third model are higher than in the empty model which is somewhat odd.

model 0 | multilevel variance 0 | MCMC 0,002
model 1 | multilevel variance 0,0053 | MCMC 0,0058
model 2 | multilevel variance 0,0062 | MCMC 0,0061
model 3 | multilevel variance 0 | MCMC 0,0007

The question is: if the variance in the normal multilevel is sometimes 0, how does this influence the MCMC and the interpretation of the results?

Thanks in advance!
Thomas

[GeorgeLeckie]

Hi Thomas,

RESPONSE TO FIRST QUESTION
DIC is a popular means of model comparison when fitting models by MCMC. Essentially, You use the statistic in the same way that you would use an AIC statistic when comparing models fitted by ML. The DIC statistic (like the AIC statistic) is penalized for model complexity, so any reduction is seen as an improvement in model fit.

RESPONSE TO SECOND QUESTION
You want to fit a cross-classified model. You first fit the a strict hierarchical model using IGLS to obtain starting values for fitting the non-hierarchical model by MCMC. The IGLS random part estimates will typically not be sensible as they have made an incorrect assumption about the structure of the data. The IGLS variances can sometimes even be stuck on zero, which is what you find. With a sufficient burn-in the model results should not be that sensitive to the starting values. However, it is prudent to check, especially when the starting values appear strange as in your case. You can play around specifying different starting values via the initsb() option in MLwiN. You can specify with different lenght burn-in periods using burnin().

I hope that helps

George

[thomasPFF]

Dear George,

First Question: Ok that is easy!

Second Question: That is also very helpful, indeed upping the burn-in periods changes the variance on the level (it goes down the higher the burn-in period) in the empty model. I will also try out different starting values, thanks!