Machine Configuration

From Marss86

Contents

Introduction

MARSS uses YAML language for all configuration files. YAML's format - can be easily read/write by Humans and Machines - makes it a perfect fit for configuring various modules and designing simulated machines for MARSS. MARSS provides some configuration files, as example, in 'config' directory. Users specify the configuration file to use via 'config' flag at compilation as shown below:

$ scons -Q config=my_machine.conf

If configuration file is not provided, then SCons uses default configuration file - 'config/default.conf' for compilation. The configuration file contains various configuration modules for cores, cache, etc. as described below.

To run a specific machine configuration give name of the machine to -macine option in your simconfig file.

To view all available machines compiled in, give 'simconfig -help' and look for 'machine' option which will list all the available machine names.

YAML

YAML is an acronym for "YAML Ain't Markup Language" which suggests that YAML is very different from other markup languages like XML. YAML is described as a human friendly data serialization language which is the main reason we have selected YAML for configuring modules of MARSS and also designing simulated machines.


Sample YAML Document

Following is a simple example of YAML document.

Fruits:
  Group1:
    - Apple
    - Banana
    - Grapes:
      - Green
      - Red
  Berries:
    - Strawberry
    - Blackberry
 
# Now we list quantities of the fruits listed above
Quantities:
    Apple: 20
    Banana: 10
    Grapes: 40oz

As you noticed, YAML consist of associative arrays and array list. In above example, 'Fruits' and 'Quantities' are top level associative array which contains arbitrary elements that may include a list, an associative array or any value. Nested elements are indented further either using space or tabs, here number of spaces or tabs is not important as long as nested elements have same left justification.

In YAML, string does not need to be enclosed by '"' which makes it much easier to read. All the comments start by '#' as in bash script.

Lists in YAML

Block representation of List

- Apple
- Banana
- Grapes

Here List is shown in a conventional block format where each element of the list is prefixed by 'hyphen+space' (- ).


Inline representation of List

[ Apple, Banana, Grapes]

For smaller lists YAML also provides 'Inline' format of List where elements are enclosed by brackets and separated by 'comma+space' (, ).

Associative Arrays

# Block format of Associative Array
fruit: Apple
quantity: 20
 
# Inline format of Associative Array
[ fruit: Apple, quantity: 20]

Associative array holds key:value pair where keys are separated by 'colon+space' (: ). As in List format, Associative array can be shown in both Block and Inline format.

More on YAML

Configuring Modules

MARSS configuration files provides 3 type of modules: Cores, Caches and Memory Controllers. For each module MARSS provides different types of base model, for example currently there are 2 core models 'Out-Of-Order' and 'Atom'. To define a module in configuration file, following format is used:

MODULE_TYPE:
    MODULE_NAME:
        base: BASE_MODULE
        # Parameters to the Module which are converted to #define
        params:
            PARAM_NAME: VALUE


MODULE_TYPE
This value can be either 'core', 'cache' or 'memory'
MODULE_NAME
This should be a unique name of the module that user is going to define.
BASE_MODULE
Each new module must have a BASE_MODULE which can be either one of the predefined modules which are part of MARSS or it can be user defined MODULE_NAME. If the BASE_MODULE is user defined MODULE_NAME, then this new module will inherit all the parameters defined in BASE_MODULE.
params
'core' and 'cache' module supports required and optional parameters.

Cores

Currently MARSS provides two Core modules 'ooo' and 'atom'. As the name suggests 'ooo' module is an out-of-order core pipeline and 'atom' is an simple in-order commit pipeline. Both these models can be configured for multi-threaded core. The default parameter values for each core can be found in respective CORE_NAME-const.h files.

A sample core module definition is shown below:

core:
    ooo_6_issue:
        base: ooo
        params:
            COMMIT_WIDTH: 4
            ISSUE_WIDTH: 6

Above, 'ooo_6_commit' core is configured with 6 ISSUE_WIDTH and 4 COMMIT_WIDTH. All other parameters are set to default. As mentioned earlier, users can create new module inherited from other user modules. So following core definition of 'ooo_6_wide' core inherits ISSUE_WIDTH of 6 from 'ooo_6_issue' and overwrites COMMIT_WIDTH to 6.

core:
    ooo_6_wide:
        base: ooo_6_issue
        params:
            COMMIT_WIDTH: 6

Cache

MARSS has 3 cache modules: wb_cache (Write-Back cache), wt_cache (Write-Through cache) and mesi_cache (MESI Coherent cache). If a cache module is defined from these base modules, then it has to provide following required parameters: SIZE, LINE_SIZE, ASSOC, LATENCY, READ_PORTS, WRITE_PORTS.

Here is a simple example of two cache modules:

cache:
    l2_1M:
        base: wb_cache
        params: # Here we need to provide all required params
            SIZE: 1M
            LINE_SIZE: 64 # in bytes
            ASSOC: 8
            LATENCY: 5
            READ_PORTS: 2
            WRITE_PORTS: 2
 
    # Here we will define 2M size L2 based on previous l2_1M
    l2_2M:
        base: l2_1M
        params: # We have inherited all required params from l2_1M
                # so we will only overwrite SIZE parameter
            SIZE: 2M

As you noticed in above example, users can define multiple modules of same type in one block, like both L2 cache modules are defined in one 'cache' block.


Memory Controllers

Currently MARSS only provide one DRAM memory controller module. Memory controller modules does not have any parameters. DRAM access delay can be configured using 'option' when the module is used in a machine. (More on this in later section on Designing Machine).

memory:
    dram_cont:
        base: simple_dram_cont

List of available modules in MARSS

Note: name in brackets is the name to use in configuration files.

Cores

  • Out-of-Order (ooo) : Supported 'option' variables are:
    • threads: Specify per core thread
  • Atom (atom) : Supported 'option' variables are:
    • threads: Configure per core threads (Note: Threading in Atom is not fully tested yet.)

Caches

  • Write-back Cache (wb_cache)
  • Write-through Cache (wt_cache)
  • MESI Coherent Cache (mesi_cache)
  • MOESI Coherent Cache (moesi_cache)

Memory Controllers

  • Simple DRAM Controller (simple_dram_cont) : Takes following 'option' variables:
    • latency: Time to access DRAM in nano-seconds

Interconnects

  • Point-2-Point (p2p)
  • Split-Phase Bus (split_bus) : Takes following 'option' variables:
    • latency: Number of cycles to broadcast
    • arbitrate_latency: Number of cycles to arbitrate between connected controllers
    • disable_snoop: Disable snoop login in bus that allows 'write-back' or 'write-through' private caches in multi-core configuration.
  • Switch (switch) : Takes following 'option' variables:
    • latency: Number of cycles to send request from source to destination

Designing A Machine

All the modules that we described earlier are now used to design a simulated machine. A simulated machine consist of one or more Core, multi-level of caches, interconnects that connect the caches and DRAM controller. Currently MARSS does not provide any simulation model for off-chip module like Bridges, PCI and PCI-E Controller etc. so the machine configuration doesn't describe them. The syntax to define a machine is shown below:

machine:
    MACHINE_NAME:
        description: MACHINE DESCRIPTION
        min_contexts: MIN_NUM_OF_CPU_CONTEXTS
        max_contexts: MAX_NUM_OF_CPU_CONTEXTS
 
        cores: # The order in which core is defined is used to assign
               # the cores in a '''machine'''
            - type: CORE_MODULE_NAME
              name_prefix: NAME_PFX
              option:
                  threads: NUM_THREADS_PER_CORE
        cache:
            - type: CACHE_MODULE_NAME
              name_prefix: NAME_PFX
              insts: NUMBER_OF_CACHE_INSTANCES
        memory:
            - type: MEMORY_MODULE_NAME
              name_prefix: NAME_PFX
              option:
                  latency: MEMORY_ACCESS_LATENCY
        interconnects:
            - type: INTERCONN_MODULE_NAME
              connections:
                  - CONT_NAME_1: INTERCONN_TYPE
                    CONT_NAME_2: INTERCONN_TYPE
                  # User will provide multiple connections here
MACHINE_NAME
A unique name of the machine
MACHINE_DESCRIPTION
Short description of the machine
MIN_NUM_OF_CPU_CONTEXTS
Minimum number of CPU Contexts allowed for this machine to build. The number of CPU Contexts to simulate is passed at compile time with 'c=NUM_CPUS' option. If number of CPU contexts is smaller than this number than this machine configuration will be ignored by compilation process.
MAX_NUM_OF_CPU_CONTEXTS
Maximum number of CPU contexts allowed for this machine to build. If number of CPU contexts is greater than this number then this machine configuration will be ignored by compilation process.
CORE_MODULE_NAME
Name of a core module which is defined in a configuration file.
CACHE_MODULE_NAME
Name of a cache module defined earlier in configuration file.
MEMORY_MODULE_NAME
Name of a memory module.
INTERCONN_MODULE_NAME
Name of interconnect module.
NAME_PFX
Name prefix used in naming each module. Name of each module is suffixed with id as number of instance of that module. For example, core that has prefix 'ooo_' and it is 2nd instance then its assigned name will be 'ooo_1'. (All suffix id starts with 0.)
INTERCONN_TYPE
This provides the type of the connection between interconnect and cache module. Available interconnect types are:
  • INTERCONN_TYPE_I : used to connect I cache to Core
  • INTERCONN_TYPE_D : used to connect D cache to Core
  • INTERCONN_TYPE_UPPER: used to connect 'Upper' interconnect to a controller in a hierarchy
  • INTERCONN_TYPE_UPPER2: used to provide second 'Upper' interconnect connection as cache modules are allowed to have two 'upper' connections.
  • INTERCONN_TYPE_LOWER: used to connect 'Lower' interconnect to a controller in a hierarhcy

Using Modules

In a machine we will declare use of various modules like cores, cache and memory. A machine must have modules from all three types. Syntax to use these modules in a machine is shown bleow:

MODULE_TYPES:
    - type: MODULE_NAME_A
      name_prefix: NAME_PFX_A
      # insts is not used for 'core' modules
      insts: INSTANCE OF THIS MODULE
      option: # This is optional
         MODULE_SPECIFIC_OPTION: Value
    - type: MODULE_NAME_B
      name_prefix: NAME_PFX_B
      insts: INSTANCE OF THIS MODULE
      option: # This is optional
         MODULE_SPECIFIC_OPTION: Value

Here each module declaration contains 'type' of module which is name of the module. 'name_prefix' is used to name each module. 'insts' tells the machine-builder how many instance of these module has to be created. This 'insts' is only required for cache and memory modules. For cores, each core can have more than one CPU context (in case of multi-threaded core), so the number of core instance is calculated based on the number of simulated CPU contexts.

To create a cache module for each core (in case of private caches) user can write '$NUMCORES' as value for 'insts'. Following is a simple example of cache module where machine builder will create same amount of cache instances as number of cores.

caches:
    - type: l1_128k
      name_prefix: L1_I_128k_
      insts: $NUMCORES

Every instance of the module is given unique name by machine builder. The name of module is created using 'name_prefix' + instance_number. For example, in above cache module if number of cores is 2 then machine builder will create two instance of l1_128k with name L1_I_128k_0 and L1_I_128k_1.


Connecting Modules

In a machine all modules are connected using various interconnects. MARSS provides two types of interconnects: Point-to-Point and Split-phase Bus. Following syntax is used to define a connection between module:

interconnects:
    - type: INTERCONNECT_MODULE
      connections:
          - MOD_X : INTERCONNECT_TYPE
            MOD_Y : INTERCONNECT_TYPE
          - MOD_A_* : INTERCONNECT_TYPE
            MOD_B : INTERCONNECT_TYPE

'interconnects' is a list where each element contains 'type' of interconnect and 'connections' where specified interconnect is used. 'type' can be either 'p2p', 'split_bus' or 'switch'. 'connections' is a list of connections between two or more modules. Each element of the list is a key-value pair where key is module instance name and value is interconnect type.

Note: For all the core modules, MARSS automatically creates CPUController which provides an interface between any core module and memory hierarchy. So to connect a core to cache, users will use 'core_X' as connection module name.

As the number of instance of modules can be dynamic based on number of simulated cores, MARSS provides two special characters '$' and '*' to connect modules with dynamic instances.


  • '$' : This special char is used to match the instance of given module that has same suffix id. Modules that uses '$' in connections must have 'insts' set to '$NUMCORES'.
  • '*' : This special char is used to connect all the instance of modules that has same name prefix as given in 'connection'.

Following is a simple example that shows how '$' and '*' is used in connecting modules.

interconnects:
    - type: p2p
      connections:
          - core_$: I
            L1_I_$: UPPER
          - core_$: D
            L1_D_$: UPPER
    - type: split_bus
      connections:
          - L1_I_*: LOWER
            L1_D_*: LOWER
            L2_0: UPPER

Here in the first connection we connect each core's CPUController to its L1_I cache using a 'p2p' interconnect. In second connection each core is connected to L1_D cache. In both these connections, for L1_I and L1_D caches, as core is in upper level in the hierarchy, 'UPPER' interconnect type is used for them. The code generated by machine builder for these connections would be like following:

  // This loop will connect L1_I cache to core
  for(i=0, i < NUM_SIM_CORES; i++) {
    p2p = new P2PInterconnect();
 
    // First connect core_$ to p2p using connection type I
    core_(i).register_interconnect(p2p, I);
 
    // Now connect L1_I_$ to p2p using connection type UPPER
    L1_I_(i).register_interconnect(p2p, UPPER);
  }
 
  // This loop will connect L1_D cache to core
  for(i=0; i < NUM_SIM_CORES; i++) {
    p2p = new P2PInterconnect();
 
    // First connect core_$ to p2p using connection type I
    core_(i).register_interconnect(p2p, I);
 
    // Now connect L1_I_$ to p2p using connection type UPPER
    L1_I_(i).register_interconnect(p2p, UPPER);
  }

For the connection defined in 'split_bus', where '*' is used for L1_I and L1_D cache all L1_I and L1_D instance will be connected to 'split_bus' interconnect. Following is a pseudo-code that will be generated by machine builder for split_bus connection.

  // This loop will connect L1_I_*, L1_D_* and L2 via split_bus
  for(i=0; i < 1; i++) {
    split_bus = new SplitPhaseBusInterconnect();
 
    // First connect all L1_I instances
    for(j=0; j < NUM_SIM_CORES; j++) {
        L1_I_(j).register_interconnect(split_bus, LOWER);
    }
 
    // First connect all L1_D instances
    for(j=0; j < NUM_SIM_CORES; j++) {
        L1_D_(j).register_interconnect(split_bus, LOWER);
    }
 
    // Now connect L2 instance
    L2.register_interconnect(split_bus, UPPER);
  }

'import' statement

MARSS allows users to split configuration into multiple files and use 'import' statement to create final configuration file. User following syntax to import configuration from other files:

import:
    - file1.conf
    - file2.conf
    - conf1/file3.conf


Sample Configuration file

Now lets put everything together and design a machine with following features:


  • Heterogeneous multicore where half of the cores are Out-of-order and other half are Atom.
  • Private L2 cache of size 256K
  • Shared L3 cache of size 4M
core:
  ooo:
    base: ooo 
    params:
      ISSUE_WIDTH: 4
      COMMIT_WIDTH: 4
 
  atom: # we use default params of atom
    base: atom
 
cache:
  l1_64k:
    base: mesi_cache
    params:
      SIZE: 64K
      LINE_SIZE: 64 # bytes
      ASSOC: 8
      LATENCY: 1
      READ_PORTS: 2
      WRITE_PORTS: 1
  l2_256k:
    base: mesi_cache
    params:
      SIZE: 256K
      LINE_SIZE: 64 # bytes
      ASSOC: 8
      LATENCY: 5
      READ_PORTS: 2
      WRITE_PORTS: 2
  l3_4M:
    base: wb_cache
    params:
      SIZE: 4M
      LINE_SIZE: 64 # bytes
      ASSOC: 8
      LATENCY: 12
      READ_PORTS: 2
      WRITE_PORTS: 2
 
memory:
  dram_cont:
    base: simple_dram_cont
 
machine:
  heterogeneous:
    description: Mix of OOO and Atom cores with private L2
    min_contexts: 2
    cores:
      - type: ooo
        name_prefix: ooo_
        option:
            threads: 1
      - type: atom
        name_prefix: atom_
        option:
            thread: 1
    caches:
      - type: l1_64k
        name_prefix: L1_I_
        insts: $NUMCORES
        option:
            private: true
      - type: l1_64k
        name_prefix: L1_D_
        insts: $NUMCORES
        option:
            private: true
      - type: l2_256k
        name_prefix: L2_
        insts: $NUMCORES
        option:
            private: true
            last_private: true
      - type: l3_4M
        name_prefix: L3_
        insts: 1
    memory:
      - type: dram_cont
        name_prefix: MEM_
        insts: 1 # Single DRAM controller
        option:
            latency: 12
    interconnects:
      - type: p2p
        connections:
          - core_$: I
            L1_I_$: UPPER
          - core_$: D
            L1_D_$: UPPER
          - L1_I_$: LOWER
            L2_$: UPPER
          - L1_D_$: LOWER
            L2_$: UPPER2
          - L3_0: LOWER
            MEM_0: UPPER
      - type: split_bus
        connections:
          - L2_*: LOWER
            L3_0: UPPER
Personal tools