How to run a dual machine network connected simulation?

[faridsamandi]

I want to run a simulation containing two independent machines that are connected together over a network (ethernet). Are there any example configuration files for it, especially using gem5's standard library? I found this sample config configs/deprecated/example/fs.py , which is similar to what I'm looking for, however, it is not using the standard library.
How can I connect two boards with ethernet and pass them to a Simulator component?

Thanks,

[powerjg]

This is not supported out of the box with the stdlib, but it should be doable with some specialization.

You can take the board that you're using (e.g., X86Board or ArmBoard) and add an ethernet device to the board. I would extend one of these (e.g., class NetworkX86Board(X86Board)) and add the ethernet device.

Then, you will need to create a new kind of board (i.e., inherit from AbstractBoard) that can take multiple "sub boards" and connect the ethernet ports together (e.g., with a switch). Then, in your new board, you will need to override connectThings and a few other functions to get everything to work. Finally, you can pass that new board (maybe called Cluster?) to the simulator and things should "just work."

Obviously, there are a lot of details to fill in here, but hopefully this gets you started.

[faridsamandi]

Thank you for the prompt and detailed explanation Jason! I’m using RiscvBoard, which already seems to have an ethernet device based on my quick look into the code. I’ll focus on creating a cluster board as you suggested. Thanks again for pointing me in the right direction!

[faridsamandi]

Hi @powerjg , I just got the chance to follow your instructions, but I’ve run into some questions early on. For the new board, say RiscvCluster, which inherits from AbstractBoard or AbstractSystemBoard as you suggested, I need to pass the board’s components (e.g., processor, etc.) during initialization. Should I 1)Initialize the new board with placeholder components that I don’t care about, and then create my two main test machines as separate RiscvBoard objects within the RiscvCluster? Or 2)Inherit RiscvCluster from RiscvBoard, treating it as one of the main machines, and then create another RiscvBoard within it, and connect it to the first board (self)?

I initially pursued the second approach (I did override _pre_instantiate and _post_instantiate methods in the RiscvCluster to call instantiations and _connect_things() on both boards) but encountered a Memory address range overlapping error during the m5.instantiate() call. I’d like to confirm if my approach is fundamentally incorrect for creating the boards, or if I’m on the right track but need to resolve additional details (e.g., handling memory for the two boards). Thanks for your help.

[powerjg]

Here is an example cluster object with an ethernet switch. I modified this some a version that I'm using for something else, and I haven't tested it at all. You'll have to define your RiscvWithEthernetBoard with the connect_switch function and custom overridden _pre_instantiate function which takes a Root instead of creates it.

class Cluster(SubSystem):
    """
    A `Cluster` can act like an `AbstractBoard` but has multiple sub-boards.
    These boards are connected together via a single ethernet switch.
    """

    def __init__(self, boards):
        super().__init__()
        self.boards = boards

        # Add your ethernet switch here
        self.switch = EtherSwitch()

        # Connect your switch to your board's ethernet ports here
        for i, board in enumerate(self.boards):
            board.connect_switch(self.switch.interface)

    def get_processor(self):
        # This is a hack for the default exit events for generators.
        return None

    def is_fullsystem(self):
        return True

    def _pre_instantiate(self, full_system=None):
        """This needs to look like an AbstractBoard so that the cluster can
        be used as a board in the simulation. The main cluster is what is
        responsible for creating the root.
        """
        from m5.objects import Root

        root = Root(full_system=True, board=self)

        # Note: The remote boards' _pre_instantiate require a root object
        # unlike the AbstractBoard
        for board in self.boards:
            board._pre_instantiate(root)

        return root

    def _post_instantiate(self):
        for board in self.boards:
            board._post_instantiate()