Combining DoParrallel and R2mlwin

Welcome to the forum for R2MLwiN users. Feel free to post your question about R2MLwiN here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to R2MLwiN: Running MLwiN from within R >> http://www.bris.ac.uk/cmm/software/r2mlwin/
Post Reply
adeldaoud
Posts: 63
Joined: Sat Aug 15, 2015 4:00 pm

Combining DoParrallel and R2mlwin

Post by adeldaoud »

Hi

I am trying to work with the doParallel package (for multicore use) and R2mlwin. However, I am not able to pass the model specification as a list object via doParallel to the runmlwin function. I believe that the problems lay in how the list is defined:

# (1) settings
library(doParallel)
library(R2MLwiN)
cl <- makeCluster(3)
registerDoParallel(cl) # register the cores

# (2) define the three models to be passed to R2mlwin via doParallel
flist<-list(c(logit(AbsolutDep, cons) ~ 1 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm),
c(logit(AbsolutDep, cons) ~ 1 + DisVic2 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm),
c(logit(AbsolutDep, cons) ~ 1 + DisVic2 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm))

result <-foreach(i=1:3, .packages="R2MLwiN", .errorhandling=c('pass')) %dopar% {
runMLwiN(flist[])
}

# (3) doParallel seems to pass the list to runMLwiN. But runMLwiN returns the following error:

> result
[[1]]
<simpleError in parse(text = x, keep.source = FALSE): <text>:2:0: unexpected end of input
1: ~
^>

[[2]]
<simpleError in parse(text = x, keep.source = FALSE): <text>:2:0: unexpected end of input
1: ~
^>

[[3]]
<simpleError in parse(text = x, keep.source = FALSE): <text>:2:0: unexpected end of input
1: ~
^>


# (4) This is how the runMLwiN function should be specified:

runMLwiN(logit(AbsolutDep, cons) ~ 1 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm)



# (5) My hypothesis is that there is something wrong in how I define the flist object above since when I initiate runMLwiN but now with the concatenate c(), I get the same error as above:

> runMLwiN(c(logit(AbsolutDep, cons) ~ 1 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm))
Error in parse(text = x, keep.source = FALSE) :
<text>:2:0: unexpected end of input
1: ~
^

# (6) I guess, therefore, it is a matter of how I am passing the list via doParrallel to rrunMLwinN. I have tried various other specifications but cannot get it to work.


Many thanks for your guidance
Adel

ChrisCharlton
Posts: 1193
Joined: Mon Oct 19, 2009 10:34 am

Re: Combining DoParrallel and R2mlwin

Post by ChrisCharlton »

The following works for me (I have also changed the example to use one of the MLwiN sample worksheets):

Code: Select all

# (1) settings
library(doParallel)
library(R2MLwiN)
data(bang)

cl <- makeCluster(3)
registerDoParallel(cl) # register the cores

# (2) define the three models to be passed to R2mlwin via doParallel
flist<-list(
  list(logit(use, cons) ~ 1 + (1 | district), D = "Binomial", estoptions = list(EstM = 0), data = bang),
  list(logit(use, cons) ~ 1 + age + (1 | district), D = "Binomial", estoptions = list(EstM = 0), data = bang),
  list(logit(use, cons) ~ 1 + age + urban + (1 | district), D = "Binomial", estoptions = list(EstM = 0), data = bang)
)

result <- foreach(i=1:3, .packages="R2MLwiN", .errorhandling=c('pass')) %dopar% {
  do.call(runMLwiN, flist[[i]])
}

stopCluster(cl)

# (3) display results

result

[[1]]

-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 
MLwiN (version: 2.35)  multilevel model (Binomial) 
          N min     mean max
district 60   3 47.78333 173
Estimation algorithm:  IGLS MQL1        Elapsed time : 0.18s 
Number of obs:  2867 (from total 2867)        The model converged after 4 iterations.
Log likelihood:      NA 
Deviance statistic:  NA 
--------------------------------------------------------------------------------------------------- 
The model formula:
logit(use, cons) ~ 1 + (1 | district)
Level 2: district     Level 1: l1id      
--------------------------------------------------------------------------------------------------- 
The fixed part estimates:  
               Coef.   Std. Err.       z    Pr(>|z|)         [95% Conf.   Interval] 
Intercept   -0.47968     0.07712   -6.22   4.982e-10   ***     -0.63084    -0.32852 
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1  
--------------------------------------------------------------------------------------------------- 
The random part estimates at the district level: 
                  Coef.   Std. Err. 
var_Intercept   0.23771     0.06404 
--------------------------------------------------------------------------------------------------- 
The random part estimates at the l1id level: 
                Coef.   Std. Err. 
var_bcons_1   1.00000     0.00000 
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 

[[2]]

-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 
MLwiN (version: 2.35)  multilevel model (Binomial) 
          N min     mean max
district 60   3 47.78333 173
Estimation algorithm:  IGLS MQL1        Elapsed time : 0.15s 
Number of obs:  2867 (from total 2867)        The model converged after 4 iterations.
Log likelihood:      NA 
Deviance statistic:  NA 
--------------------------------------------------------------------------------------------------- 
The model formula:
logit(use, cons) ~ 1 + age + (1 | district)
Level 2: district     Level 1: l1id      
--------------------------------------------------------------------------------------------------- 
The fixed part estimates:  
               Coef.   Std. Err.       z    Pr(>|z|)         [95% Conf.   Interval] 
Intercept   -0.47850     0.07741   -6.18   6.361e-10   ***     -0.63023    -0.32678 
age          0.01331     0.00429    3.10    0.001941   **       0.00489     0.02173 
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1  
--------------------------------------------------------------------------------------------------- 
The random part estimates at the district level: 
                  Coef.   Std. Err. 
var_Intercept   0.23992     0.06435 
--------------------------------------------------------------------------------------------------- 
The random part estimates at the l1id level: 
                Coef.   Std. Err. 
var_bcons_1   1.00000     0.00000 
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 

[[3]]

-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 
MLwiN (version: 2.35)  multilevel model (Binomial) 
          N min     mean max
district 60   3 47.78333 173
Estimation algorithm:  IGLS MQL1        Elapsed time : 0.17s 
Number of obs:  2867 (from total 2867)        The model converged after 5 iterations.
Log likelihood:      NA 
Deviance statistic:  NA 
--------------------------------------------------------------------------------------------------- 
The model formula:
logit(use, cons) ~ 1 + age + urban + (1 | district)
Level 2: district     Level 1: l1id      
--------------------------------------------------------------------------------------------------- 
The fixed part estimates:  
                Coef.   Std. Err.       z    Pr(>|z|)         [95% Conf.   Interval] 
Intercept    -0.65405     0.07647   -8.55   1.204e-17   ***     -0.80394    -0.50417 
age           0.01413     0.00434    3.26     0.00112   **       0.00563     0.02263 
urbanUrban    0.67889     0.09496    7.15   8.741e-13   ***      0.49277     0.86502 
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1  
--------------------------------------------------------------------------------------------------- 
The random part estimates at the district level: 
                  Coef.   Std. Err. 
var_Intercept   0.19309     0.05539 
--------------------------------------------------------------------------------------------------- 
The random part estimates at the l1id level: 
                Coef.   Std. Err. 
var_bcons_1   1.00000     0.00000 
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 
Note: the simulation example in the development version of R2MLwiN has an example of using doParallel (see https://github.com/rforge/r2mlwin/blob/ ... CGuide08.R). Also, if you choose to run more than one MCMC chain in the development version then these will be run in parallel via doParallel.

adeldaoud
Posts: 63
Joined: Sat Aug 15, 2015 4:00 pm

Re: Combining DoParrallel and R2mlwin

Post by adeldaoud »

Chirs, this is brilliant! Thank you. It works as advertised.

I will have a look at the development version for running more than one MCMC chain. That could be quite useful to speed up things.

Cheers
Adel

Post Reply