Checkpoints in MARSS

From Marss86

MARSS uses QEMU's snapshot support in QCOW2 disk image to create checkpoints and run simulations from checkpoints. MARSS also uses PTLsim's MMIO based 'ptlcall' functions to communicate between System Under Test (system in Virtual Machine) and MARSS. This page contains information on how to use 'ptlcalls' to start or stop between simulation and emulation, create checkpoints at specific points in benchmarks etc.

PTLcall Functions

Based on PTLsim's ptlcall methods, MARSS uses MMIO based technique to communicate between SUT and MARSS. When a guest system is booted, MARSS registers a small MMIO region in guest system's memory space via QEMU. Once the guest system is booted, writing to or reading from the MARSS's MMMIO memory area will trigger specific functionalities in MARSS which is used to change various status of the simulator.

In the source code, we provide 'ptlcalls.h' file which implements various PTLcalls API functions listed below:

  • ptlcall(W64 Operation, W64 Arg1, W64 Arg2, W64 Arg3, W64 arg4, W64 Arg5, W64 Arg6) : This function is a base function which will check if the operating-system supports PTLcalls or not and will make a MMIO read/write to a PTLCall specific memory region to communicate with simulator.
  • ptlcall_multi(char* list[], size_t length, int flush) : This function will enqueue multiple simulation commands as string and send it to the simulator
  • ptlcall_multi_enqueue(char* list[], size_t lenght) : This function will only enqueue the simulation commands but will not send it to the simulator
  • ptlcall_multi_flush(char* list[], size_t length) : This function will enqueue simulation commands and flush them (or send them) to the simulator
  • ptlcall_single(char* command, int flush) : This function will add a single command to the simulator's queue and optionally flush it if 'flush == 1'
  • ptlcall_single_enqueue(char* command) : This function will add a single command to the simulator's queue and does not flush it to the simulator
  • ptlcall_single_flush(char* command) : This function will add a single command to the simulator's queue and flush it to the simulator
  • ptlcall_nop() : This function will send "-run" command to the simulator
  • ptlcall_switch_to_sim() : This function will send "-run" command to the simulator
  • ptlcall_switch_to_native() : This function will send "-native" command to the simulator
  • ptlcall_kill() : This function will send "-kill" command to the simulator
  • ptlcall_capture_stats(char* snapshot_name) : This function will send a "-snapshot-now %snapshot_name" and "-run" to the simulator to take a snapshot of the stats and save in the stats output file.
  • ptlcall_checkpoint_generic(char* name, int action) : A generic function to create a checkpoint with given name and execute an action provided as argument. Action can be any of the following:
    • PTLCALL_CHECKPOINT_AND_CONTINUE
    • PTLCALL_CHECKPOINT_AND_SHUTDOWN
    • PTLCALL_CHECKPOINT_AND_REBOOT
    • PTLCALL_CHECKPOINT_AND_PAUSE
    • PTLCALL_CHECKPOINT_DUMMY
  • ptlcall_checkpoint_and_continue(char* name) : Create a checkpoint at current state of the VM and continue in emulation mode.
  • ptlcall_checkpoint_and_shutdown(char* name) : Create a checkpoint at current state of the VM and shutdown the VM.
  • ptlcall checkpoint_and_reboot(char* name) : Create a checkpoint at current state of VM and reboot the VM.
  • ptlcall_checkpoint_and_pause(char* name) : Create a checkpoint at current state of VM and pause the VM.
  • ptlcall_checkpoint() : Create a checkpoint at current state of the VM with name of the checkpoint as "default", and shutdown the VM.
  • ptlcall_checkpoint_dummy() : This function will only pause the VM at current state and will not create any checkpoint.

Using PTLcalls

As described in above section, PTLcalls provide various API function to communicate between SUT and simulator. Users can integrate these functions into the benchmark code or create a simple binary that calls one of the PTLcall functions to switch between emulation to simulation mode or change simulation config parameters etc.

Here is a simple C code that uses 'ptlcall_switch_to_sim()' function to switch from emulation to simulation in SUT.

#include "ptlcalls.h"
#include <stdio.h>

int main() {

	printf("Switching to simulation\n");
	ptlcall_switch_to_sim();
	return 0;
}

Compile the code as 'start_sim' binary and you can switch to simulation mode right before you start you benchmark as shown below:

$ ./start_sim ; ./[your_benchmark] [benchmark_options] 

To stop the simulation after your benchmark as completed, you can use either call 'ptlcall_switch_to_native()' or 'ptlcall_kill()'. If you use 'ptlcall_switch_to_native()' it will simply switch from simulation to emulation mode and will not kill the SUT. While if you use 'ptlcall_kill()' it will stop the simulation, save all the stats and logs and will kill the SUT.

Following code shows how to use 'ptlcall_kill()' and kill the simulation once your benchmark is completed.

#include "ptlcalls.h"
#include <stdio.h>

int main() {

	printf("Stopping simulation\n");
	ptlcall_kill();
	return 0;
}

Compile above code as 'kill_sim' binary and you can stop the simulation once your benchmarks completes as shown below:

$ ./start_sim; ./[your_benchmark] [benchmark_options] ; ./kill_sim

See the section below how you can configure your simulation parameters like log file name, stats file name, etc.

Configure MARSS When Using PTLcalls

When a PTLcall api function is called from the SUT, if the simulation parameters are not configured, the simulator will stop the SUT so user can configure the simulation parameters and start it. But the best practice is to configure the basic simulation parameters like log file name, stats file name etc before the PTLcall is made.

For example, when you use 'start_sim' and 'kill_sim' based method shown above to simulate your benchmark, before you give this command in SUT, switch to QEMU's monitor and give following command to configure simulation parameters.

(qemu) simulate -logfile bench.log -stats bench.stats -loglevel 0 

Once you configured your simulation parameters, switch to SUT's console and give the simulation command shown above:

$ ./start_sim; ./[your_benchmark] [benchmark_options] ; ./kill_sim

After the 'start_sim' binary make a 'ptlcall_switch_to_sim()', the simulator will start running and will generate stats in bench.stats file once it receives 'ptlcall_kill()' from 'kill_sim' binary.

Personal tools