The Yellow Dot on rmf_schedule_visualizer

[simk0024]

I wonder how the robot trajectories (yellow dot and green path) are calculated on RMF Schedule Visualizer.
When executing rmf_demos, most of the time yellow dot is align with the position of robot on rviz.
However, when I running physical robot, the yellow dot might not aligned, sometimes move ahead of robot or sometimes behind.
For physical robot, both path and current location of robot are published in /robot_state.

Besides, I also notice when the yellow dot is behind robot, robot/adaptertends to respond slower, i.e. lift_request or door_request published only when yellow dot reach the pre-door/pre-lift waypoint, instead of published when the robot reach.
(This might be my mistake)

How to make sure the yellow dot is always aligned with robot pose?

[mxgrey]

What kind of fleet adapter is your physical robot using? Free Fleet + Full Control? Custom Full Control? Traffic Light? Knowing how your robot is integrated will help narrow down any possible misuse of the API.

When you see the yellow dot trailing or leading the location of the physical robot, does it ever snap back to (roughly) the robot's actual location, or does it keep trailing/leading indefinitely?

How significant is the trailing or leading? Within a second of time difference? Several seconds? Longer?

Usually significant trailing or leading will be caused by a low update frequency or high latency. The rmf_demos will naturally have almost no trailing or leading because they're updating their states at a very high frequency, and since the simulation is on the same machine as the fleet adapter, there's no network latency.

RMF is designed to be robust to low update frequencies and high latency, so the traffic management should still generally be able to work in the presence of these issues, just maybe not quite as efficiently. So depending on what degree of performance you care about, the trailing/leading might be okay or you might need to identify the bottleneck/issue in your update pipeline.

[simk0024]

I am using Free Fleet + Full Control.

The performance is not stable, sometimes it aligned but sometimes it doesn't.
Worst cases I have encountered are

1. robot was stuck, but yellow dot reached goal
2. robot reached goal, but yellow dot not moving.

IS that the yellow dot updated based on current pose published by robot?
Does published path affect the yellow dot?
What is the RMF desired update frequency? (currently robot publishing at 1Hz)

[mxgrey]

IS that the yellow dot updated based on current pose published by robot?
Does published path affect the yellow dot?

Using Free Fleet + Full Control, the yellow dot will be based on both the current location and the remaining path. The function that you can see here uses that information to predict when the robot will arrive at its next target waypoint (not the final goal) based on where it currently is. The fleet adapter uses the difference between that live prediction and the original planned estimate to determine a time shift in the schedule, which will move the yellow dot forward or backward along the path.

What is the RMF desired update frequency? (currently robot publishing at 1Hz)

1 Hz is a reasonable update frequency. If you have an update frequency of 1Hz, then I would expect to see the yellow dot snapping into place each second. If that's not what you're seeing then I'm not sure what would be happening. As far as I know, the if the purple robot state marker is getting updated, then the yellow schedule marker should also be getting snapped into place for the free fleet.

robot was stuck, but yellow dot reached goal

This can easily happen if the robot stops submitting state updates. However, that shouldn't happen just because the robot is stuck (assuming by "stuck" you mean there are obstacles in the way). This makes me wonder if robot state updates are getting dropped on your network?

robot reached goal, but yellow dot not moving.

This is especially surprising to me. That yellow schedule marker will stay in motion unless there is something actively holding it in place.

If you're able to share screen capture videos of these issues happening in rviz, that might help me wrap my head around the situation.

[simk0024]

This can easily happen if the robot stops submitting state updates. However, that shouldn't happen just because the robot is stuck (assuming by "stuck" you mean there are obstacles in the way). This makes me wonder if robot state updates are getting dropped on your network?

Yes, you are right, there was obstacle blocking the robot path.
Here is the video we took video
The lift is mock lift (just a lift entrance), robot was moving at same level.
If you look at the last 30s of video, robot was blocked but the yellow dot keeps going to the goal. (Sorry that we don't have the full video.)

Of course, this only happened once. Most of the time there was minor trailing/leading.
Like what you mentioned, network might be the factor causing this. If robot current pose is published correctly, yellow dot should always aligned with the purple marker, right?

[mxgrey]

I think the minor leading/trailing in that video is within a normal and acceptable range. But clearly something is going quite wrong at the end when the scheduled route (yellow) flies away from the robot's actual state.

If robot current pose is published correctly, yellow dot should always aligned with the purple marker, right?

One thing that's tricky with this visualization is that since the robot state isn't moving, I can't be entirely sure that it is being updated. We might just be seeing a residual marker of robot's last known location in the rviz visualization, and the system might not be receiving any new robot states while the robot is sitting there.

When the robot sits there as the schedule marker flies away, is the robot supposed to be waiting for the lift to change floors and/or open its doors, or is the robot supposed to be moving by then?

• If the robot is supposed to be moving, then I would suspect the connection between the robot and the fleet adapter (and/or free fleet) has been lost somehow.
• If the robot is supposed to continue waiting for the lift, then that would indicate some kind of bug in the fleet adapter LiftRequest phase. As you can see here the robot's scheduled itinerary should be getting delayed while the robot waits for the lift.

I'll keep thinking about this to see if I can come up with any debugging recommendations.