www.cmm.bristol.ac.uk/forum

Hi there!
I’m running a multilevel logistic model (MCMC) with levels of walking for recreation (none, low, medium and high) allowing a level 1 predictor (gender) to be random across neighbourhoods (i.e. random coefficient) and then adjusting for a cross-level interaction (gender with a level 2 predictor). How could I test for a significant reduction in the random coefficient beyond noise? For instance, would a Deviance or Wald test be appropriate?

Many thanks,

Fatima.

1 Use a Wald test of whether a parameter or a set of parameters are zero – a univariate Wald and a multivariate Wald (for multiple simultaneous testing) are available under Interval and Tests; in the main you need to be careful with random effects estimates but the fixed ones are generally fine;

2 Use Customised predictions to calculate the predictions for specific values of the predictors and their associated confidence intervals – these can then be displayed and inspected – this is particularly helpful if you are modelling interactions.

3 Use the DIC statistic which can be used to compare the goodness of fit of models (including discrete outcome ones) – this assesses the badness of fit penalized for model complexity (the Pd) – the penalty is estimated during the MCMC process - so this one is appropriate fro your question

We have developed a lot of resources for understanding and using the software and this is the best place to start:
http://www.bristol.ac.uk/cmm/software/m ... urces.html

On testing and IGLS/ RIGLS see Chapter on significance testing in C&Hvol1: Jones, K and Subramanian, V S (2014) Developing multilevel models for analysing contextuality, heterogeneity and change using MLwiN, Volume 1 , University of Bristol.

On the DIC statistic see
Browne, W.J. (2012) MCMC Estimation in MLwiN, Centre for Multilevel Modelling, University of Bristol.

And for an extended account of discrete outcome models (including DIC and customised predictions) see
C&Hvol2: Jones, K and Subramanian, V S (2013) Developing multilevel models for analysing contextuality, heterogeneity and change using MLwiN, Volume 2, University of Bristol.

It is quite reasonable as you put in predictors into a discrete outcome model that the higher level random part can go up ( because the level 1 random part - eg the binomial) cannot go down. This is covered in detail in the Lemma training materials in Lemma Modules 6 & 7
LEMMA: Multilevel modelling online course