Discussion on Next Generation OpenRMF's simulation pipeline

[TanJunKiat]

I want to open up a discussion on the matter of simulations for OpenRMF as we transit into the next iteration or generation of middleware.

Background

The current OpenRMF relies heavily on Gazebo for its simulations. While convenient, Gazebo runs very heavily on physics, which might not be fully utilized.

The default robotic fleets work on a very simple point-to-point style of navigation, while doors and lifts have a simple lift controller to serve as a proxy to test and showcase the features of OpenRMF.

Transition to Next Generation

As we go forward to the next iteration, we can actually look at what we can and want to do with OpenRMF simulations.

With the site editor progressing steadily, we have two possible paths that we can take in terms of simulations.

The first is to have simulations embedded into the site editor, making the site editor a powerful application that comprises all the elements of OpenRMF, from map definition to the OpenRMF’s core and simulations.

The second is to have a separate pipeline that allows users to run simulations.

There are some merits to either solution, but we can start laying down some common groundwork while discussing to see which option is better.

Ideas for common scope

Item	Description	Remarks
bevy_infrastructure	A Bevy library that contains common infrastructure elements like doors and lifts	This can be helpful when designing a simulation for OpenRMF where doors and lifts integrations are one of the focus.
A robot fleet simulator on Bevy	Potentially a Bevy library that allows users to add robots with a simulated navigation stack that moves around Bevy-defined entities.
Workcell integration	Potentially a Bevy package that allows users to add workcells with basic motion planning capabilities	The focus should be on creating visual representations of the workcells while allowing interaction with OpenRMF. We should utilize existing crates to perform robot control, like motion planning and joint control.

PS: I will try to update the main thread periodically as the discussion grows, so please do point out any misinformation or confusion in the main thread.

[arjo129]

My 2 cents:

I think its great to have point-to-point simulation in bevy (aka Event driven simulation). We can definitely handle lifts and doors without the full physics from gazebo. However, there is still some value in providing SDF worlds that get exported to gazebo. For instance, if an end user wants to do E2E testing with FreeFleet, it makes sense to build on gazebo's existing infrastructure (we don't have an example for this, but it'd definitely be nice to have). Similarly if I want to integrate a non-rmf based system with rmf, (for instance some new type of sensor like a Lidar system for detecting obstructions) gazebo still has more sensor models for us to use.

Workcell integration

Motion planning is a huge can of worms. While we can use prerecorded paths in bevy (perhaps building on k and open-rr's work), I can imagine that this will be a non-trivial amount of work. We really should build on other peoples work for this rather than re-inventing the wheel.

[TanJunKiat]

I think its great to have point-to-point simulation in bevy (aka Event driven simulation). We can definitely handle lifts and doors without the full physics from gazebo. However, there is still some value in providing SDF worlds that get exported to gazebo. For instance, if an end user wants to do E2E testing with FreeFleet, it makes sense to build on gazebo's existing infrastructure (we don't have an example for this, but it'd definitely be nice to have). Similarly if I want to integrate a non-rmf based system with rmf, (for instance some new type of sensor like a Lidar system for detecting obstructions) gazebo still has more sensor models for us to use

Maybe we can explore to see if we can have the best of both worlds (event-based using Bevy and physics-based with Gazebo).

Hopefully, the design of the next OpenRMF can be configurable to adapt to either simulator.

For the event-based, in theory, it should be as straightforward as adding the simulator plugin into the Bevy application.

For Gazebo, we can opt for ROS or Zenoh to get and pull information from Gazebo.

Motion planning is a huge can of worms. While we can use prerecorded paths in bevy (perhaps building on k and open-rr's work), I can imagine that this will be a non-trivial amount of work. We really should build on other peoples work for this rather than re-inventing the wheel.

Yes, I agree with not re-inventing the wheel. The focus of the event-based simulation should be purely the interaction between the simulator and OpenRMF, while providing visual representation of the states of the arm.