Combining DoParrallel and R2mlwin

[adeldaoud]

Hi

I am trying to work with the doParallel package (for multicore use) and R2mlwin. However, I am not able to pass the model specification as a list object via doParallel to the runmlwin function. I believe that the problems lay in how the list is defined:

# (1) settings
library(doParallel)
library(R2MLwiN)
cl <- makeCluster(3)
registerDoParallel(cl) # register the cores

# (2) define the three models to be passed to R2mlwin via doParallel
flist<-list(c(logit(AbsolutDep, cons) ~ 1 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm),
c(logit(AbsolutDep, cons) ~ 1 + DisVic2 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm),
c(logit(AbsolutDep, cons) ~ 1 + DisVic2 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm))

result <-foreach(i=1:3, .packages="R2MLwiN", .errorhandling=c('pass')) %dopar% {
runMLwiN(flist[])
}

# (3) doParallel seems to pass the list to runMLwiN. But runMLwiN returns the following error:

> result
[[1]]
<simpleError in parse(text = x, keep.source = FALSE): <text>:2:0: unexpected end of input
1: ~
^>

[[2]]
<simpleError in parse(text = x, keep.source = FALSE): <text>:2:0: unexpected end of input
1: ~
^>

[[3]]
<simpleError in parse(text = x, keep.source = FALSE): <text>:2:0: unexpected end of input
1: ~
^>


# (4) This is how the runMLwiN function should be specified:

runMLwiN(logit(AbsolutDep, cons) ~ 1 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm)



# (5) My hypothesis is that there is something wrong in how I define the flist object above since when I initiate runMLwiN but now with the concatenate c(), I get the same error as above:

> runMLwiN(c(logit(AbsolutDep, cons) ~ 1 + (1 | country) , D = "Binomial", estoptions = list(EstM = 0), data = dfsm))
Error in parse(text = x, keep.source = FALSE) :
<text>:2:0: unexpected end of input
1: ~
^

# (6) I guess, therefore, it is a matter of how I am passing the list via doParrallel to rrunMLwinN. I have tried various other specifications but cannot get it to work.


Many thanks for your guidance
Adel

[ChrisCharlton]

The following works for me (I have also changed the example to use one of the MLwiN sample worksheets):

Code: Select all

# (1) settings
library(doParallel)
library(R2MLwiN)
data(bang)

cl <- makeCluster(3)
registerDoParallel(cl) # register the cores

# (2) define the three models to be passed to R2mlwin via doParallel
flist<-list(
  list(logit(use, cons) ~ 1 + (1 | district), D = "Binomial", estoptions = list(EstM = 0), data = bang),
  list(logit(use, cons) ~ 1 + age + (1 | district), D = "Binomial", estoptions = list(EstM = 0), data = bang),
  list(logit(use, cons) ~ 1 + age + urban + (1 | district), D = "Binomial", estoptions = list(EstM = 0), data = bang)
)

result <- foreach(i=1:3, .packages="R2MLwiN", .errorhandling=c('pass')) %dopar% {
  do.call(runMLwiN, flist[[i]])
}

stopCluster(cl)

# (3) display results

result

[[1]]

-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 
MLwiN (version: 2.35)  multilevel model (Binomial) 
          N min     mean max
district 60   3 47.78333 173
Estimation algorithm:  IGLS MQL1        Elapsed time : 0.18s 
Number of obs:  2867 (from total 2867)        The model converged after 4 iterations.
Log likelihood:      NA 
Deviance statistic:  NA 
--------------------------------------------------------------------------------------------------- 
The model formula:
logit(use, cons) ~ 1 + (1 | district)
Level 2: district     Level 1: l1id      
--------------------------------------------------------------------------------------------------- 
The fixed part estimates:  
               Coef.   Std. Err.       z    Pr(>|z|)         [95% Conf.   Interval] 
Intercept   -0.47968     0.07712   -6.22   4.982e-10   ***     -0.63084    -0.32852 
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1  
--------------------------------------------------------------------------------------------------- 
The random part estimates at the district level: 
                  Coef.   Std. Err. 
var_Intercept   0.23771     0.06404 
--------------------------------------------------------------------------------------------------- 
The random part estimates at the l1id level: 
                Coef.   Std. Err. 
var_bcons_1   1.00000     0.00000 
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 

[[2]]

-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 
MLwiN (version: 2.35)  multilevel model (Binomial) 
          N min     mean max
district 60   3 47.78333 173
Estimation algorithm:  IGLS MQL1        Elapsed time : 0.15s 
Number of obs:  2867 (from total 2867)        The model converged after 4 iterations.
Log likelihood:      NA 
Deviance statistic:  NA 
--------------------------------------------------------------------------------------------------- 
The model formula:
logit(use, cons) ~ 1 + age + (1 | district)
Level 2: district     Level 1: l1id      
--------------------------------------------------------------------------------------------------- 
The fixed part estimates:  
               Coef.   Std. Err.       z    Pr(>|z|)         [95% Conf.   Interval] 
Intercept   -0.47850     0.07741   -6.18   6.361e-10   ***     -0.63023    -0.32678 
age          0.01331     0.00429    3.10    0.001941   **       0.00489     0.02173 
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1  
--------------------------------------------------------------------------------------------------- 
The random part estimates at the district level: 
                  Coef.   Std. Err. 
var_Intercept   0.23992     0.06435 
--------------------------------------------------------------------------------------------------- 
The random part estimates at the l1id level: 
                Coef.   Std. Err. 
var_bcons_1   1.00000     0.00000 
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 

[[3]]

-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 
MLwiN (version: 2.35)  multilevel model (Binomial) 
          N min     mean max
district 60   3 47.78333 173
Estimation algorithm:  IGLS MQL1        Elapsed time : 0.17s 
Number of obs:  2867 (from total 2867)        The model converged after 5 iterations.
Log likelihood:      NA 
Deviance statistic:  NA 
--------------------------------------------------------------------------------------------------- 
The model formula:
logit(use, cons) ~ 1 + age + urban + (1 | district)
Level 2: district     Level 1: l1id      
--------------------------------------------------------------------------------------------------- 
The fixed part estimates:  
                Coef.   Std. Err.       z    Pr(>|z|)         [95% Conf.   Interval] 
Intercept    -0.65405     0.07647   -8.55   1.204e-17   ***     -0.80394    -0.50417 
age           0.01413     0.00434    3.26     0.00112   **       0.00563     0.02263 
urbanUrban    0.67889     0.09496    7.15   8.741e-13   ***      0.49277     0.86502 
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1  
--------------------------------------------------------------------------------------------------- 
The random part estimates at the district level: 
                  Coef.   Std. Err. 
var_Intercept   0.19309     0.05539 
--------------------------------------------------------------------------------------------------- 
The random part estimates at the l1id level: 
                Coef.   Std. Err. 
var_bcons_1   1.00000     0.00000 
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- 

Note: the simulation example in the development version of R2MLwiN has an example of using doParallel (see https://github.com/rforge/r2mlwin/blob/ ... CGuide08.R). Also, if you choose to run more than one MCMC chain in the development version then these will be run in parallel via doParallel.

[adeldaoud]

Chirs, this is brilliant! Thank you. It works as advertised.

I will have a look at the development version for running more than one MCMC chain. That could be quite useful to speed up things.

Cheers
Adel