Number of groups varies between 2- and 3-level multivariate MLM

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
Post Reply
erikruzek
Posts: 16
Joined: Tue May 21, 2019 3:57 pm

Number of groups varies between 2- and 3-level multivariate MLM

Post by erikruzek »

Hi all,

I am running a multivariate MLM that has repeated observations (level 1) of teachers (level 2) within schools (level 3). The data has 550 non-missing observations on the two outcomes for 65 teachers in 23 schools.

Variable | Obs Mean Std. Dev. Min Max
-----------------+---------------------------------------------------------
ELA_overall | 550 2.472286 .7009207 .5555556 3.888889
CLASS_over~l | 550 4.394666 .6439461 2.739683 6.023016


There was not an occasion level variable, so I created one that does not repeat across teachers using

Code: Select all

sort id 
gen occ_c = _n 
The two level model is as follows:

Code: Select all

sort id occ_c
runmlwin (ELA_overall cons, eq(1)) (CLASS_overall cons, eq(2)), ///
	level2(id: (cons, eq(1)) (cons, eq(2))) ///
	level1(occ_c: (cons, eq(1)) (cons, eq(2))) nopause correlations //gives you corrs vs. covariances
The number of groups is as follows (total number of obs = 550):

Code: Select all

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
       id       |       65        1            8.5         17
-----------------------------------------------------------

I then run the 3-level model, asking for residuals at each level:

Code: Select all

sort sch1 id occ_c
runmlwin (ELA_overall cons, eq(1)) (CLASS_overall cons, eq(2)) , ///
	level3(sch1: (cons, eq(1)) (cons, eq(2)), residuals(uS)) ///
	level2(id: (cons, eq(1)) (cons, eq(2)), residuals(uT)) ///
	level1(occ_c: (cons, eq(1)) (cons, eq(2)), residuals(r)) nopause correlations //gives you corrs vs. covariances
This model reports an additional two id groups, and the number of observations per id differs slightly (total number of obs = 550):

Code: Select all

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
      sch1     |       23       1            23.9         60
      id       |       67       1              8.2         17
-----------------------------------------------------------
Any ideas what might be going on?

Thanks,
Erik
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Number of groups varies between 2- and 3-level multivariate MLM

Post by ChrisCharlton »

This would normally happen if the same teacher id occurred within more than one school id. What happens if you run the first model with the data sorted as in the second model?
erikruzek
Posts: 16
Joined: Tue May 21, 2019 3:57 pm

Re: Number of groups varies between 2- and 3-level multivariate MLM

Post by erikruzek »

Thanks, Chris. You identified the problem exactly. I have two teachers who each taught in two different schools. Technically that means teachers and schools are cross-classified. However, with only two of 65 teachers in this situation, I'm curious whether you think it makes sense to model the cross-classification?
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Number of groups varies between 2- and 3-level multivariate MLM

Post by ChrisCharlton »

I can't really advise on that (although a quick search turned up https://doi.org/10.3389/fpsyg.2016.00328, which might be relevant), technically you should probably fit the cross-classified model but it may not change the results much. Chapter 18 of the replication materials for the MLwiN user guide (see https://www.bristol.ac.uk/cmm/software/ ... /examples/) demonstrates fitting a cross-classified model with IGLS, but it is easiest to set up if you use MCMC instead. Maybe try running the models as both nested and cross-classified and see what differences it makes to the results?
erikruzek
Posts: 16
Joined: Tue May 21, 2019 3:57 pm

Re: Number of groups varies between 2- and 3-level multivariate MLM

Post by erikruzek »

Thanks, Chris. Knowing the CC exists and it is so easily handled in MLwiN, I agree it makes sense to model it correctly. I did so and variance and covariance estimates at each of the levels are quite similar, and as a result, the number of groups being reported is exactly aligned with expectations.
Post Reply