Level 2 residuals for cross-classified logistic regression model

Welcome to the forum for R2MLwiN users. Feel free to post your question about R2MLwiN here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to R2MLwiN: Running MLwiN from within R >> http://www.bris.ac.uk/cmm/software/r2mlwin/
Post Reply
fionap123
Posts: 4
Joined: Tue Jul 04, 2017 7:58 pm

Level 2 residuals for cross-classified logistic regression model

Post by fionap123 »

Hello.

I have been fitting a cross-classified logistic regression model using MCMC for a binary response variable and would like to extract the estimated level 2 residuals and match these to the level 2 group names. However, the residuals I get from using the resi.store=TRUE option in R2MLwiN are not the same as those from running the same model in MLwiN itself. The residuals are larger in R2MLwiN and I wondered if there are raw rather than shrunken? If I use "reflate" in resioptions, the residuals are the same as without this option.

I am also unsure how to match the level 2 residuals to the level 2 group names using R2MLwiN (this can be done in MLwiN using 'Sort' from the data manipulation menu).

Thanks,

Fiona.
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Level 2 residuals for cross-classified logistic regression model

Post by ChrisCharlton »

The residuals for MCMC models should always be the mean of the "u" parameter chains and therefore (assuming that the chosen random number generator and seed are the same) should give the same results both ways you run the model. The reflate option does not apply to MCMC models, which is why the values would be the same with this turned on. If you run the model from R2MLwiN with debugmode turned on you can abort the macro prior to returning to R and request the residuals there. This should then be equivalent to running the model directly in MLwiN, allowing you to compare.

If the results are different then it's possible that there is a bug in the code being generated by R2MLwiN. In that case if you were able to provide an example where this occurs I would be able to investigate further.
fionap123
Posts: 4
Joined: Tue Jul 04, 2017 7:58 pm

Re: Level 2 residuals for cross-classified logistic regression model

Post by fionap123 »

Thanks for your reply. I have run the model in debugmode and managed to compare the residuals and also open the relevant temp .dta file. It seems as though the R2MLwiN output is multiplying the level 2 residuals by 2.

Is there something in particular that I could attach or provide to help investigate this further?

Thanks,

Fiona.
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Level 2 residuals for cross-classified logistic regression model

Post by ChrisCharlton »

If you could provide a reproducible example with syntax using publicly available data that demonstrates the problem then I can try to track down the cause. If you are unable to do this then it would be useful to see the generated MLwiN macro files in case I can find a problem there.
fionap123
Posts: 4
Joined: Tue Jul 04, 2017 7:58 pm

Re: Level 2 residuals for cross-classified logistic regression model

Post by fionap123 »

Hi Chris.

I ran a slightly modified version of one of the exercises from the R2MLwiN JSS article (Zhang et al., 2016, p.26-27) as follows:

library(R2MLwiN)
data("bang1")
F6 = logit(use, cons) ~ 1 + age + lc + urban + (1|district)
binomialMCMC <- runMLwiN(Formula = F6, D = "Binomial", data = bang1, estoptions = list(debugmode=TRUE, show.file=TRUE, resi.store=TRUE, EstM = 1))
binomialMCMC@residual$lev_2_resi_est_Intercept

The residuals printed by the final command are double the level 2 residuals produced directly in MLwiN.

Thanks,

Fiona.

Zhang, Z., Parker, R., Charlton, C.M., Browne, W.J., 2016. R2MLwiN: A package to run MLwiN from within R. Journal of Statistical Software 72, 1–43.
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Level 2 residuals for cross-classified logistic regression model

Post by ChrisCharlton »

Thank you very much for this example, I believe that I have now tracked down the cause. There is a typo in the code for loading the residuals into R that means that instead of looking for and loading standardised residuals it will instead add the unstandardised residuals estimates to themselves. I will fix this in the next R2MLwiN release, however in the mean time the work around is to request residual chains via resi.store.levs and take the means yourself.

Sorry for any inconvenience.
fionap123
Posts: 4
Joined: Tue Jul 04, 2017 7:58 pm

Re: Level 2 residuals for cross-classified logistic regression model

Post by fionap123 »

Thanks for the investigation and reply. Would it be ok to use:

binomialMCMC@residual$lev_2_resi_est_Intercept<-binomialMCMC@residual$lev_2_resi_est_Intercept/2

to replace the level 2 residuals? I would like to be able to produce some plots using caterpillarR.
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Level 2 residuals for cross-classified logistic regression model

Post by ChrisCharlton »

Yes, this should be fine. Note that if you have specified multiple chains then the incorrect residuals would have been used when applying Ruben's rules to the standard errors and variances, and therefore these may also be incorrect.
Post Reply