www.cmm.bristol.ac.uk/forum

Posted: **Thu Apr 17, 2014 2:08 pm**

Dear forum members,

After a long struggle with a MCMC cross-classified multilevel model that finally worked by using runmlwin the paper that contains it is finally accepted. However, the editor asked a simple question. Is the P value one sided or two sided?

After some searching on this site and I am still confused so wanted to pose the question here just to be certain as it seems the p value should be interpreted different from regular regression. To make it a little more general:

- How is the p value measured and how should it be interpreted? Is it one sided or two sided?

A second question, equally small, is that I find in papers using this method and program, some of them report the SD and others report the Cred. Intervals. Is there, from a runmlwin-perspective a most sensible choice?

All the best,
Thomas

Below I copy-pasted the log of the Model:

Code: Select all

. runmlwin ln_gecorrigeerdeprijs year vertaling beperkteoplage heruitgave ///
    gedicht literatuur romantisch scifi andergenre lt lr grootte grootte2 hardcover paginas cons, ///
    level3(uitgeverij1: cons) ///
    level2(auteur1: cons) ///
    level1(nummer: cons) ///
    mcmc(cc) initsmodel(m1igls)nopause
       
      MLwiN 2.28 multilevel model                     Number of obs      =     60435
      Normal response model
      Estimation algorithm: MCMC
      
      -----------------------------------------------------------
                      |   No. of       Observations per Group
       Level Variable |   Groups    Minimum    Average    Maximum
      ----------------+------------------------------------------
          uit1 |     1083          1       55.8       4041
              au11 |    16202          1        3.7        376
      -----------------------------------------------------------
      
      Burnin                     =        500
      Chain                      =       5000
      Thinning                   =          1
      Run time (seconds)         =        907
      Deviance (dbar)            =    6223.16
      Deviance (thetabar)        =     320.81
      Effective no. of pars (pd) =    5902.35
      Bayesian DIC               =   12125.51
      ------------------------------------------------------------------------------
      ln_gecorrigeerdeprijs|Mean  Std. Dev.     ESS     P       [95% Cred. Interval]
      -------------+----------------------------------------------------------------
              year |  -.0049567   .0001821     1641   0.000    -.0053111   -.0045981
         vertaling |   .0487968   .0043725      977   0.000     .0404417    .0573221
      beperkteop~e |   .2537362   .0162274     1196   0.000     .2219473    .2853769
        heruitgave |  -.2247376   .0030407     1971   0.000    -.2308626   -.2186812
           gedicht |   .0331743   .0068281     1363   0.000      .019811    .0464638
        literatuur |   .0781542   .0052405     1168   0.000     .0680186    .0883305
        romantisch |  -.0847578    .008602      729   0.000    -.1014977   -.0674954
             scifi |  -.0439112   .0106811     1046   0.000    -.0647463   -.0233281
        andergenre |   .0125445   .0055338     1434   0.011     .0018639    .0236906
                lt |   .0380802   .0076082     2586   0.000     .0233335    .0526477
                lr |   .0493877    .009969     2081   0.000     .0302323    .0690403
           grootte |   .1495113   .0033265     1961   0.000     .1429073    .1559708
          grootte2 |  -.0011875   .0000714     2271   0.000    -.0013268   -.0010483
         hardcover |   .1811161   .0036141     1855   0.000     .1739914    .1882009
           paginas |   .0009432   9.29e-06     2925   0.000     .0009249    .0009613
              cons |  -.0867717   .0413735     1395   0.017    -.1687535   -.0063418
      ------------------------------------------------------------------------------
      
      ------------------------------------------------------------------------------
         Random-effects Parameters |     Mean   Std. Dev.   ESS     [95% Cred. Int]
      -----------------------------+------------------------------------------------
      Level 3: uit1         |
                         var(cons) |  .1800995  .0088095   2956   .1640432  .1982508
      -----------------------------+------------------------------------------------
      Level 2: au1             |
                         var(cons) |  .0142907  .0004455    303    .013433  .0151593
      -----------------------------+------------------------------------------------
      Level 1: nummer              |
                         var(cons) |  .0648978  .0004215   2585   .0640611  .0657223
      ------------------------------------------------------------------------------

Posted: **Thu Apr 17, 2014 2:46 pm**

Hi Thomas,

The p-value which is reported by default is a one-sided Bayesian p-value. It is the proportion of the chain which is of the opposite sign to the point estimate.

You can obtain z-ratio and frequentist two-sided p-values and 95% confidence intervals (based on assuming normal sampling distributions) by specifying the z-ratio option. Bayesians would of course frown at this, hence this not being the default option.

Whether you report SD or 95% Bayesian credible Intervals is up to you. I guess the latter are more informative when you have skewed sampling distributions.

Best wishes

George

Posted: **Wed Jun 18, 2014 6:43 pm**

I am a bit puzzled how the quote in Leckie's paper fits in this (regarding to binary response models):

The means and standard deviations of the sampled parameters from the monitoring period are used as
parameter estimates and standard errors while the 2.5th and 97.5th quantiles of these chains
provide Bayesian 95% credible intervals, analogous to 95% condence intervals. When vague
or diffuse prior distributions are specified (the default in MLwiN), the parameter estimates
are essentially maximum likelihood estimates. The MLwiN MCMC Manual provides an ac-
cessible introduction to MCMC methods, in the context of the MLwiN software. Technical
details are given in Goldstein (2011).

runmlwin - A program to run the MLwiN multilevel modelling software from within stata in Journal of Statistical Software

Am I right that, assuming default MLwiN prior distributions, the 95% credible intervals can be interpreted as classic 95% confidence intervals (i.e. similar interpretations to Maximum Likelihhood estimates)?

Posted: **Thu Jun 19, 2014 11:44 am**

Hi Funnyyyy,

The MCMC and ML points estimates and standard errors will tend to be very similar when models are fitted to large datasets with vague priors (MLwiN default priors are vague). Many researchers therefore interpret their MCMC results as they would their ML results.

Best wishes

George

www.cmm.bristol.ac.uk/forum

interpreting P values

interpreting P values

Re: interpreting P values

Re: interpreting P values

Re: interpreting P values