Hi,
I have just noticed that the way I order the variables in my multilevel model (repeated measures) can lead to very small but slight adjustments in
1. Log likelihood
2. Parameter estimates (plus std errors)
3. The number of iterations to convergence.
It is a negligible difference but it still bothers me. Should this be happening?
Many thanks
Michael
Order of variables?
-
GeorgeLeckie
- Site Admin
- Posts: 432
- Joined: Fri Apr 01, 2011 2:14 pm
Re: Order of variables?
Hi Michael,
That sounds odd and it would be good to get to the bottom of it.
Could you post the runmlwin output for your model twice, once specified each way so we can take a look at the differences
Best wishes
George
That sounds odd and it would be good to get to the bottom of it.
Could you post the runmlwin output for your model twice, once specified each way so we can take a look at the differences
Best wishes
George
-
michaellawton
- Posts: 11
- Joined: Tue Nov 22, 2011 10:40 pm
Re: Order of variables?
Hi here is one model:
Here is another model with the same data but the (independent) variables listed in a different order:
Code: Select all
. runmlwin score cons fp1 fp6 ageonset fp1ageonset fp6ageonset relapse_var gender , level2(patid: cons fp1 fp6) level1(newids: cons fp1 ) nopause maxiterations(1250)
MLwiN 2.24 multilevel model Number of obs = 1872
Normal response model
Estimation algorithm: IGLS
-----------------------------------------------------------
| No. of Observations per Group
Level Variable | Groups Minimum Average Maximum
----------------+------------------------------------------
patid | 406 1 4.6 26
-----------------------------------------------------------
Run time (seconds) = 1.04
Number of iterations = 13
Log likelihood = -2569.0691
Deviance = 5138.1382
------------------------------------------------------------------------------
score | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
cons | 1.323798 .6758441 1.96 0.050 -.0008321 2.648428
fp1 | .1227989 .0882338 1.39 0.164 -.0501361 .2957339
fp6 | -.4999869 .5871845 -0.85 0.394 -1.650847 .6508735
ageonset | .0356616 .0174237 2.05 0.041 .0015117 .0698115
fp1ageonset | .0020272 .0028564 0.71 0.478 -.0035714 .0076257
fp6ageonset | .0102426 .0178876 0.57 0.567 -.0248164 .0453016
relapse_var | .1746091 .0902812 1.93 0.053 -.0023389 .351557
gender | -.3471408 .1971454 -1.76 0.078 -.7335387 .0392572
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: patid |
var(cons) | 3.16518 .4899186 2.204958 4.125403
cov(cons,fp1) | -.2643705 .0606884 -.3833175 -.1454235
var(fp1) | .0532954 .0116529 .0304561 .0761348
cov(cons,fp6) | -.3283667 .4002215 -1.112786 .4560531
cov(fp1,fp6) | -.23276 .074663 -.3790967 -.0864232
var(fp6) | 2.305654 .5452258 1.237031 3.374277
-----------------------------+------------------------------------------------
Level 1: newids |
var(cons) | .5237606 .0411513 .4431054 .6044157
cov(cons,fp1) | -.0066677 .0024093 -.0113899 -.0019455
var(fp1) | .0001007 .0001186 -.0001317 .0003332
------------------------------------------------------------------------------Here is another model with the same data but the (independent) variables listed in a different order:
Code: Select all
. runmlwin score cons ageonset fp1ageonset fp6ageonset fp1 fp6 relapse_var gender , level2(patid: cons fp1 fp6) level1(newids: cons fp1 ) nopause maxiterations(1250)
MLwiN 2.24 multilevel model Number of obs = 1872
Normal response model
Estimation algorithm: IGLS
-----------------------------------------------------------
| No. of Observations per Group
Level Variable | Groups Minimum Average Maximum
----------------+------------------------------------------
patid | 406 1 4.6 26
-----------------------------------------------------------
Run time (seconds) = 0.89
Number of iterations = 11
Log likelihood = -2569.0776
Deviance = 5138.1553
------------------------------------------------------------------------------
score | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
cons | 1.322037 .6758115 1.96 0.050 -.0025289 2.646604
ageonset | .0356989 .0174239 2.05 0.040 .0015486 .0698491
fp1ageonset | .0020431 .0028527 0.72 0.474 -.003548 .0076343
fp6ageonset | .0101395 .0178706 0.57 0.570 -.0248861 .0451652
fp1 | .1221076 .0881018 1.39 0.166 -.0505687 .294784
fp6 | -.4957939 .586572 -0.85 0.398 -1.645454 .6538662
relapse_var | .1746632 .0902913 1.93 0.053 -.0023046 .351631
gender | -.3471212 .1971438 -1.76 0.078 -.7335159 .0392735
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: patid |
var(cons) | 3.166923 .4898657 2.206804 4.127042
cov(cons,fp1) | -.2643535 .060627 -.3831803 -.1455267
var(fp1) | .0531886 .0116431 .0303685 .0760087
cov(cons,fp6) | -.3291243 .40044 -1.113972 .4557237
cov(fp1,fp6) | -.23223 .0746553 -.3785518 -.0859082
var(fp6) | 2.303365 .5456064 1.233996 3.372734
-----------------------------+------------------------------------------------
Level 1: newids |
var(cons) | .523794 .0411529 .4431358 .6044522
cov(cons,fp1) | -.0066686 .0024094 -.0113908 -.0019463
var(fp1) | .0001007 .0001186 -.0001317 .0003332
-------------------------------------------------------------------------------
ChrisCharlton
- Posts: 1354
- Joined: Mon Oct 19, 2009 10:34 am
Re: Order of variables?
One possible reason for this behaviour is that the two models are converging from different directions and are considered to have converged before reaching exactly the same point.
The default tolerance in MLwiN (and runmlwin) is 10e-2, which corresponds to a 1% change in the value of the estimate from one iteration to the next. If all parameters for an iteration are within this tolerance from the previous iteration then the model is considered to have converged.
You can test whether this is the reason for the behaviour you are seeing by setting the tolerance() estimation option to a higher number and seeing whether the two sets of results become closer.
The default tolerance in MLwiN (and runmlwin) is 10e-2, which corresponds to a 1% change in the value of the estimate from one iteration to the next. If all parameters for an iteration are within this tolerance from the previous iteration then the model is considered to have converged.
You can test whether this is the reason for the behaviour you are seeing by setting the tolerance() estimation option to a higher number and seeing whether the two sets of results become closer.