Batch Runs

From Marss86

NOTE: This guide referes to run_bench.py in 'features' branch.

Once you have setup MARSS, made changes for your studies, created all checkpoints for your workloads then all you need is to run simulations and generate results. MARSS provides a small utility 'run_bench.py' in 'util' folder that helps running multiple simulations in batch mode. This is a simple python script that uses configuration file (default is 'util/util.cfg') to read your simulation setup and runs one or more simulation instances.

run_bench.py Commandline Options

This script is in 'utils/' folder and should be executed from the top MARSS directory. In most of the cases you won't need to modify this script and all your simulation setup parameters are read from a configuration file. The command line options for this script are:

  • -h, --help Show the help message
  • -d OUTPUT_DIR, --output-dir=OUTPUT_DIR The output directory where you want to store all log and results files.
  • -c CONFIG, --config=CONFIG The configuration file, default is 'util/util.cfg' file
  • -e, --email Send email once the simulations are complete. It uses 'xoauth.txt' file
  • -i ITERATE, --iterate=ITERATE Run each checkpoint/benchmark for N times
  • -n N, --num-insts=N Number of simulation instances to run in parallel
  • --chk-names=C1,C2 Comma separated list of checkpoints to run which overrides the list of checkpoints from config file

As described in the options above, use -d DIR_NAME to save all log and stats file in that folder. Log file and stats file are prefixed based on configuration which is explained in detailed in next section. Use '-i' option to run multiple iterations of each benchmark. The script will save each iteration in 'run_1' to 'run_N' folders.

Also run_bench.py automatically adds '-snapshot' to the qemu command line so all disk images are treated as read-only.

Configuration Format

By default 'run_bench.py' uses 'util.cfg' file to read the configuration and list of checkpoints to run. You can override this with '-c' commandline option to use other configuration file. The configuration file is in INI format. A sample configuration file is shown below:

[DEFAULT]
marss_dir = /your/local/marss/directory
util_dir = %(marss_dir)s/util
img_dir = /your/disk/image/dir
qemu_bin = %(marss_dir)s/qemu/qemu-system-x86_64
default_simconfig = -kill-after-run -quiet
 
[email]
xoauth = xoauth.txt
to = youremail@domain.com
 
[suite spec-int]
checkpoints = gcc, bzip2, sphinx, mcf
 
[run spec-int-ooo]
suite = spec-int
images = %(img_dir)s/spec2006.qcow2
memory = 2G
simconfig = -logfile %(out_dir)s/%(bench)s.log
  -stats %(out_dir)s/%(bench)s.yml
  %(default_simconfig)s
  -machine ooo

As shown in the sample file above, the DEFAULT section contains various default parameters. These parameters can be used in any section via reference mechanism shown in run spec-int-ooo section's images parameter. The DEFAULT section declares parameters like MARSS directory, util directory use for finding supported utility scripts, img_dir contains disk images, qemu_bin contains which binary to use for simulation runs.

A suite SUITE_NAME section declare the list of checkpoints to use for simulation run. Each suite section must have checkpoints variable which is a comma separated list of checkpoints in the disk image.

A run RUN_NAME section contains various variables used by 'run_bench.py' script to run simulation in batch mode. This section must contain these variables: suite, images, memory, simconfig.

  • A suite refers to the SUITE_NAME section declared earlier in configuration file, which contains list of checkpoints to use. A single suite can be referenced in multiple run sections.
  • The images contains a list of disk images to use for the simulations. MARSS now support read-only mode of disk images, so only specify single disk image in images. To run multiple instances of simulation in parallel, use -n command-line option in run_bench.py.
  • A memory specify how much amount of RAM used for the VM.
  • The simconfig specify various simulation configuration parameters. The last parameter, qemu_args, specifies extra arguments that you want to provide to simulation command.
  • Optional qemu_args variable can be used to specify extra command-line arguments to simulator.

Running Simulations

To run simulation with 'run_bench.py' script first populate your configuration file as shown in previous section. In example above, section 'run spec-int-ooo' specify the configuration to run checkpoints in 'suite spec-int' using disk images specified in 'images' parameter. Use run_bench.py as shown below:

  $  ./util/run_bench.py spec-int-ooo

Above command will run simulations for each checkpoint specified in 'spec-int' suite and it will store the results files in the directory where 'run_bench.py' was started. Use -d switch to specify output directory:

  $  ./util/run_bench.py -d run/spec_int_results spec-int-ooo

To run multiple instances of simulations in parallel use -n switch:

  $  ./util/run_bench.py -d run/spec_int_results -n 4 spec-int-ooo

If you want to run each simulation for more than once, use '-i' switch:

  $  ./util/run_bench.py -d run/spec_int_results -n 4 -i 3 spec-int-ooo  # It will run each checkpoint for 3 times

Simulations takes time and you don't want to be waiting for them to complete and its a hassle to keep checking if they are finished or not. So we provide a small emailing script 'send_gmail.py' to send email via 'gmail'. run_bench.py uses this script with '-e' switch to send email when all of your simulations are finished so you can relax and enjoy reddit without switching to console to look if simulations are over or not.

  $  ./util/run_bench.py -d run/spec_int_results -n 4 -i 3 -e spec-int-ooo
Personal tools