www.cmm.bristol.ac.uk/forum

Posted: **Mon May 14, 2012 6:03 pm**

Dear George,

I am currently using your runmlwin module to run a multi-level logistic regression model, but I am having a bit of difficulty with model diagnostics. I seem to be able to obtain BLUPS, but my attempts to get standardized residuals, leverage, influence (Cook's D?) and deletion residuals have been unsuccessful. In addition, I don't seem to be able to generate my lowest level residuals.

Below is a copy of the code I am currently using with a fair bit of success and obtaining my BLUPS at the pen and animal levels:

xi:runmlwin y x cons,level3(pen:cons, residuals(stub3)) level2(id:cons, residuals(stub2)) level1(obs) discrete(distribution(binomial) link(logit) denominator(cons) pql1) nopause or rigls

Do you have any suggestions for obtaining the other residual diagnostics and level1 residuals? I have tried typing "(stub3, std)", but the option is not recognized. Similarly, the addition of the residuals option for level1 has been unsuccessful.

Thank you very much for any assistance.

Sincerely,

David

Posted: **Mon May 14, 2012 6:29 pm**

Hi David,

Thanks for your post.

Here is an example of how to get standardised, leverage, influence and deletion residuals at level-2 in addition to the standard residuals that one normally retrieves.

Code: Select all

use http://www.bristol.ac.uk/cmm/media/runmlwin/bang, clear
generate lc1 = (lc==1)
generate lc2 = (lc==2)
generate lc3plus = (lc>=3)
runmlwin use cons lc1 lc2 lc3plus age, level2(district: cons, residuals(u, standardised leverage influence deletion)) level1(woman) discrete(distribution(binomial) link(logit) denominator(cons)) nopause
egen pickone_district = tag(district)
sum u0* if pickone_district==1

Below are the same commands again, but this time with the Stata output

Code: Select all

. use http://www.bristol.ac.uk/cmm/media/runmlwin/bang, clear

. generate lc1 = (lc==1)

. generate lc2 = (lc==2)

. generate lc3plus = (lc>=3)

. runmlwin use cons lc1 lc2 lc3plus age, level2(district: cons, residuals(u, standardised leverage influence deletion)) level
> 1(woman) discrete(distribution(binomial) link(logit) denominator(cons)) nopause
 
MLwiN 2.25 multilevel model                     Number of obs      =      2867
Binomial logit response model
Estimation algorithm: IGLS, MQL1

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
       district |       60          3       47.8        173
-----------------------------------------------------------

Run time (seconds)   =       1.69
Number of iterations =          5
------------------------------------------------------------------------------
         use |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        cons |  -1.367111   .1233779   -11.08   0.000    -1.608927   -1.125294
         lc1 |   .9899751   .1264296     7.83   0.000     .7421777    1.237773
         lc2 |   1.275233   .1381648     9.23   0.000     1.004435    1.546031
     lc3plus |   1.215676   .1424506     8.53   0.000     .9364783    1.494874
         age |  -.0187756   .0062489    -3.00   0.003    -.0310232   -.0065281
------------------------------------------------------------------------------

------------------------------------------------------------------------------
   Random-effects Parameters |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: district            |
                   var(cons) |   .2740886   .0713836      .1341794    .4139979
------------------------------------------------------------------------------

. egen pickone_district = tag(district)

. sum u0* if pickone_district==1

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
          u0 |        60    2.38e-08    .4370753  -.8447976   1.138374
        u0se |        60    .2926989    .0633727   .1707095   .4847279
       u0std |        60   -.0074523    1.015855  -2.098604   2.465453
       u0lev |        60     .185646    .1000665   .0546547   .6221709
       u0del |        60   -.0050843    1.029522  -2.163041   2.581039
-------------+--------------------------------------------------------
       u0inf |        60    .1052886    .0814914   .0019182   .3148382

These different types of residuals are explained in Chapter 15 of the MLwiN User Manual

http://www.bristol.ac.uk/cmm/software/m ... man-09.pdf

The following may also be useful, although it is a while since I have read it

http://www.springerlink.com/content/978 ... =1&locus=0

Also see Chapter 6 of the Leyland and Goldstein (2001) Multilevel Modelling of Health Statistics if you have a copy

In terms of level-1 residuals in logit models. Remember that logit model are expressed in terms of the mean (actually the log odds or logit of the mean) rather than the observed binary response. The model equation therefore does not contain a level-1 residual in the same way that there is a level-1 residual in continuous response models. You can of course think of the difference between the observed binary response and the predicted probability as being the level-1 residual. If you wanted to obtain this, you could calculate this by first predicting the predicted probability for each woman, and then subtracting this predicted probability from her observed response.

Code: Select all

predict xb
gen xbu = xb + u0
gen p = invlogit(xbu)
gen lev1residual = use  - p

I hope this helps

Best wishes

George

www.cmm.bristol.ac.uk/forum

Re: Residuals for multi-level logistic model in runmlwin

Re: Residuals for multi-level logistic model in runmlwin

Re: Residuals for multi-level logistic model in runmlwin