Random intercept vs. random slopes in 2LevelImpute

Welcome to the forum for Stat-JR users. Feel free to post your question about Stat-JR here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

We will add further support to the Stat-JR website, such as FAQs and tutorials, as soon as it is available; the Stat-JR website can be found here: http://www.bristol.ac.uk/cmm/software/statjr/
shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by shakespeare »

So I wanted to provide a little context here. Leaving aside ivh for the moment and considering other variables, you can see on the attached document some of the problems I'm having. Unlike MLwin, there are fewer options for adjustment in the imputation template, so any direction you can provide would be appreciated. The first set of diagnostics are for the temperature variable (tcodec). The trace plot is a little too spikey, but the density looks pretty decent. I ran 5000 iterations between imputations X 10 imputations for a total of 50000. I'm not sure if the Raftery-Lweis diagnostic is suggesting 12522 between imputations. My understanding is that is the suggested run for a single data set, how this applies to MI is not clear to me. Also, temperature is formatted XX.X degrees C, so I'm not sure how many significant digits I need to specify, but that could suggest a run that is so long that it is not practical. In any event, the effective sample size is pretty small, so I'm going to have to increase the length of the run, I'm just not sure how much.

The next variable is race (racerec), a four category variable. It's very hard to interpret the trace since everything is so compressed on the x axis. This is another variable that will require a longer run given the small effective sample size, but the huge negative number in the Raftery_Lewis diagnostic is puzzling.

The last variable is a binary variable denoting singleton/multiple births. This looks pretty decent to me, although The trace plot is unusual.

Anyway, suggestions about adjustments would be appreciated.
Attachments
TreeIvhMCMCDiagnostics2.doc
(117.5 KiB) Downloaded 746 times
richardparker
Posts: 61
Joined: Fri Oct 23, 2009 1:49 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by richardparker »

Hi - sorry, my earlier post may have caused some confusion here: looking at your diagnostic plots, it looks like you're plotting the imputed variables themselves, which would explain their curious diagnostics (e.g. tcodec is a continous variable; racerec has four categories; bncat is a binary variable).

The chains themselves can be found in an outputted dataset called 'out'. Within that, you will find a column called 'chain' which you can use as an indicator to chop up the dataset and run diagnostics for individual chains, if you wish.

Best wishes,

Richard
shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by shakespeare »

I'm not sure I'm following this. The file out.dta has 127 betas, but each has only has 10 observations. It seems like these chains should be much longer. And how would I use the chain variable to divide the sample? Nothing in the MLwin manual seems to shed any light and I can't get the MCMCColumnDiagnostics to yield any output.
richardparker
Posts: 61
Joined: Fri Oct 23, 2009 1:49 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by richardparker »

Hi - the chains for each parameter in this file will only be as long as the number of iterations you have requested between imputations (which must have been 10?) The 2LevelImpute template will take the first iteration post-burnin for each chain run (the number of chains it runs depends on the number of processors available and the number of requested datasets), and then another after the requested interval (up to the number it needs to reach the number of datasets requested, although all - including those it doesn't use - appear in the 'out' file).

So in this example there's very short chain lengths (10) for each parameter in the 'out' file. The diagnostics the 2LevelImpute template automatically returns are derived in different ways: it returns separate trace plots (on the same graph) for each chain, separate kernel density plots (on same graph) for each chain, it joins together the chains for the ACF, PACF and MCSE, but treats the chains separately for the BGRD (which is a multiple chains diagnostic); it also adds together the ESS value for each chain to derive the ESS value returned in the results.

So, given your inputs, there is only a small portion of each chain in this particular example; this may nevertheless help you detect large discrepancies, although as a general point, an interval of 10 iterations is very short, and naturally there is a higher risk of auto-correlation between iterations a few intervals apart than between iterations many more intervals apart.

Best wishes,

Richard
shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by shakespeare »

I never was able to make sense of the chains in my ivh model, so I'll return to that later. I've run a random intercept model on a binary outcome which runs to completion. It generates a lot of output which I think should be something that could be displayed in a popout window according to the documentation. I'm primarily interested in the imputation model equations and the plots of the chains. I can see a .txt file for the equations (Imputation_Model_equation.txt) and many .svg files (e.g. Imputation_Model_beta1_0.svg) for chains in the imputation model. If I select one of these files and click popout, a new browser tab is opened, but I never see any output displayed. Am I doing this right? If so, whay am I not seeing anything? Thx.
richardparker
Posts: 61
Joined: Fri Oct 23, 2009 1:49 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by richardparker »

Hi - the .svg files can take a short time to generate when you request to view them. I've just tested the 2LevelImpute template, and the .svg plots did successfully appear - including via 'popout' - but only after I waited a short time, so that might be the issue? (Stat-JR generates them 'on demand' to keep down the computational overheads which occur by default.)

By Imputation_Model_equation, did you mean a .tex file (rather than a .txt file)? Imputation_Model_equation.tex is one of the outputs, but this doesn't actually contain an equation (it will just say "No equation specified for this template") - it's just an output which automatically appears when models are run via Stat-JR: it's not populated with an equation for every template - i.e. for some it serves as a placeholder for potential future functionality.
shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by shakespeare »

Yes, that is a .tex extension. Thanks for clarifying this is not yet populated. That's a nice feature in REALCOM. It would be high on my wish list.

So I tried the patient approach with one of the .svg files (Imputation_Model_beta1_0.svg). I've been waiting about 1.5 hours, but nothing appears on the new tab generated by clicking popout. Any suggestions?
richardparker
Posts: 61
Joined: Fri Oct 23, 2009 1:49 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by richardparker »

Hi - yes, 1.5 hours is an excessively long time: can I just ask what browser (including version number) you're using, please?

Also note that an alternative means of viewing the .svg files is to download the outputs (via the green Download button, just above where the outputs are displayed in the Stat-JR TREE interface); again this will take time (it took a good few minutes for me - you may note a flurry of activity in the command line window as it generates all the outputs, including .svg files). You'll then get a zipped folder, which you can extract as normal (Windows' in-house zip extractor complained about a file type for me, althought it still extracted fine; I got no complaints returned when extracting via 7-Zip).
shakespeare
Posts: 70
Joined: Thu Feb 14, 2013 11:12 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by shakespeare »

Using IE10. I'm in a 64 bit operating environment if that makes any difference.

After running my model, I downloaded the files by clicking the green download button and saving the zip file to disk. I then extracted the files from the archive using 7-Zip-this produced a lot individual files, including the .svg files. It's these files that I'm locating in the results pane and attempting to display in the popout window. I think this is the procedure that you have described below.
richardparker
Posts: 61
Joined: Fri Oct 23, 2009 1:49 pm

Re: Random intercept vs. random slopes in 2LevelImpute

Post by richardparker »

Hi - we've tested Stat-JR using Chrome and Firefox, much less so with IE, so there may be issues specific to that browser - unfortunately I haven't been able to test the action you were trying to perform with IE10 here, so can't quite see whether it's a browser issue or not. That's good that you can access the .svg files via other means; you can also run via Chrome / Firefox in future too, which might help - assuming you've access to them.
Post Reply