Shell scripts (Linux/Mac-specific)

Now let us write a script that will run 10 such simulations in parallel. Create a file called batch-sim with the following contents:

#!/bin/bash
for ((i = 1; i <= 10; i++))
do
  Rscript sim.R &
done
wait

This is a simple loop that calls Rscript sim.R ten times. One key remark is the & at the end of the Rscript line: this tells the shell to start the job in the background and immediately move to the next iteration of the loop, so all ten simulations run simultaneously. The wait at the end holds the script open until all background jobs have finished.

Before we can run the above command, we have to tell Linux that the file is an executable script. This is accomplished with the chmod command:

chmod u+x batch-sim

The u+x means that we want to give the user (that’s us!) permission to execute the file.

We can now run all 10 simulations with a single command:

./batch-sim

This will create 10 files like sim-a3f9bx2k.rds, sim-7hq2mn4p.rds, etc. FYI: the ./ in front of batch-sim tells the shell to look in the current directory for the command (otherwise the system only looks in the folders specified by the $PATH environment variable).

To combine them in R (same as on the previous page):

files <- list.files(pattern="^sim-.*\\.rds$")
p <- do.call(c, lapply(files, readRDS))