runmlwin + mi estimate: Starting values and Residuals

[LorenaOrtega]

Dear all,

I am using runmlwin together with mi estimate to run my models and combine the results from 5 imputed datasets and it is working really well for me (thank you ever so much CMM team!). I just have a couple of very basic questions:

1) When I store the parameters from the combined results, estimated using IGLS, and try to use them as starting values for my MCMC estimation I get the following error message: "The inits model option is invalid: No parameters in the stored model match those specified in the current model".
This is part of the code I am trying to run:

Code: Select all

*IGLS
mi estimate, vartable cmdok dots: ///
runmlwin SPAN CONS GRSTART GRSTART2 if (STCOHORT==1), ///
level2 (STUDID: CONS GRSTART, residuals(u)) ///
level1(GRADE: CONS) nopause forcesort

estimates store model1spancoh1igls

*** MCMC
mi estimate, vartable cmdok dots: ///
runmlwin SPAN CONS GRSTART GRSTART2 if STCOHORT==1, ///
level2 (STUDID: CONS GRSTART, residuals(u)) ///
level1(GRADE: CONS) ///
mcmc(burnin(1000) chain(10000)) initsmodel(model1spancoh1igls)  nopause forcesort

So I tried using the starting values from the model estimated using IGLS in only one of the imputations instead, and that does work, but I was wondering whether it is possible and advisable to start the MCMC estimation with the values from the combined IGLS results.

2) When I use mi estimate, residuals are not created/stored in the dataset. I was wondering whether combined residuals can be created and stored so I can use them for residual graphs and prediction lines, as using the residuals obtained from only one of the imputation would not be as accurate. Do you have any suggestions?

Many thanks!!!
Lorena

[ChrisCharlton]

1) The initsmodel option is looking for the matrices e(b) and e(V) from the stored model, however the -mi- command saves the estimates into matrices named e(b_mi) and e(V_mi), which is why they cannot be found. You should be able to work around this by assigning these to new matrices, i.e.

Code: Select all

matrix b = e(b_mi)

Code: Select all

matrix V = e(V_mi)

and then passing these to the command with the initsb() and initsv() commands.

The starting values should just be giving the model a sensible place to start, and if it is run long enough it should eventually get to the correct distribution regardless of what they are set to. If you provide starting values that are further away then it might take more iterations for the model to converge on the correct distribution.

2) I suspect that as the -mi- command is already changing the data for each successive model it is running it isn't preserving any data written back by the command, and hence what you are seeing. The only workaround that I can think of would be to manually run the models again in a loop independently of the -mi- command, saving the residuals generated for each model.

[LorenaOrtega]

Thank you very much, Chris! That makes a lot of sense.

However, I am afraid I am running into a different problem now: When I assign e(b_mi) and e(V_mi) to new matrices and use them as starting values for the MCMC estimation through the initsb() and initsv() commands, I get an error message: "Warning: variance matrix is nonsymmetric or highly singular".
I know this happens when one of the random part parameter matrices has negative variances or covariance parameters which imply correlations which lie outside (-1,1). The problem disappears when I use initsb() only. So I think I am failing to see what initsv() brings in terms of staring values given that initsb() already produces starting values for the random parameters that seem to work fine in my case.
Is it ok to use initsb() only?

Thanks again for all your help!

[GeorgeLeckie]

Hi Lorena,

Think intsv() is only used to help tune MH algorithm proposal distribution (i.e., the 'adapting' period you see when using the MLwiN GUI to fit models using MCMC). As long as your burn-in is sufficiently long (judge this by looking at convergence diagnostics in the usual way) it shouldn't really matter that you don't specify initsv().

Best wishes

George

[GeorgeLeckie]

Hi Lorena,

In terms of your original question, as long as you specify a sufficiently long burn-in so that your monitoring iterations are no longer influenced by your starting values (check convergence diagnostics), it shouldn't really matter whether you specify a common set of starting values for the five imputed datasets or dataset specific values. Hopefully you should see this by obtaining effectively the same results when you do your analysis each way.

Best wishes

George

[LorenaOrtega]

Hi George,

Thank you very much for your prompt response! This is extremely helpful!
The only issue I am struggling with at the moment is obtaining MCMC convergence diagnostics and residuals when using runmlwin and mi estimate together. I will try to implement Chris's suggestion for combining residuals. In terms of the convergence diagnostics, I know this depends on my missing data and the quality of the multiple imputation itself, but would you say that the convergence diagnostics obtained from one of my imputations are likely to be similar to those from the other imputations so that reporting one set would suffice? And in terms of model fit, reporting the average DIC across imputations should be ok, right?

Thanks again for all your useful suggestions!

Best wishes,
Lorena

[ChrisCharlton]

To obtain MCMC diagnostics information for each model used by -mi- you can take the same approach as with the residuals, i.e. run the models again in a loop, following each with an -mcmcsum- command to convert the chains into variables with the getchains option, which you can then save to a .dta file. After running the models you can then retrieve each chain from the files and analysis it using -mcmcsum- with the variables option.

[GeorgeLeckie]

Hi Lorena,

Yes look at the convergence diagnostics for all five imputed datasets, but assuming they are similar enough, just report one set and say that the others are available upon request.

Best wishes

George

[LorenaOrtega]

Thank you very much, George and Chris!
This is really helpful!

Best wishes,
Lorena