Message versioning and ROS 2 message translation

[GuillaumeLaine]

This documents the newly versioned subset of PX4 messages, and the ROS 2 translation node that enables seamless communication between PX4 and ROS 2 applications that may be using different message definition versions.

Related Work

• [PX4-Autopilot] Implementing ROS 2 translation node and CI checks: Message versioning and translation for ROS PX4-Autopilot#24113
• [PX4-Autopilot] Introducing versioned PX4 messages: msg: introduce subset of versioned messages  PX4-Autopilot#23850
• [px4_msgs] Updating build paths for versioned messages: CMakeLists: include versioned msg path px4_msgs#46

To Do

• Complete TODOs in the markdown of this PR

[GuillaumeLaine]

@hamishwillee I think the PR is in a review ready state. I left a few TODOs in the markdown to resolve before merging. Mostly things that I need to go back to once the last related PR is merged

[hamishwillee]

This is a little messy and perhaps can be simplified. In particular

• ROS2 Message Translation node supports versioning, not the versioning supporting the translation node.

Perhaps something like:

Suggested change

	Message versioning was introduced to address the challenges of maintaining compatibility across systems where different versions of message definitions may be in use, such as between PX4 and external systems like ROS 2 applications.
	This versioning mechanism supports the [ROS 2 Message Translation Node](../ros2/px4_ros2_msg_translation_node.md), which enables seamless communication between PX4 and ROS 2 applications; when different versions of message definitions are in use, the ROS 2 translation node ensures that messages can be converted and exchanged correctly.
	Message versioning has been in introduced in PX4 v1.16 to make it easier to maintain compatibility between PX4 and ROS 2 versions.
	This versioning mechanism uses the [ROS 2 Message Translation Node](../ros2/px4_ros2_msg_translation_node.md) to convert between different versions of messages, allowing seamless communication between PX4 and ROS 2 applications; when different versions of message definitions are in use.

Perhaps add a diagram showing where the magic happens?

It would also be good "somewhere here" to mention any forward thinking about the ROS 2 native translation system that they have been talking about for a couple of years. Perhaps as a note at the end.

[GuillaumeLaine]

Good idea, I've added a diagram and a note concerning ROS native translation.

ROS2 Message Translation node supports versioning, not the versioning supporting the translation node.

I may be thinking of this wrong, but in my mind message versioning works as a standalone feature (it's just about some messages having a version number incremented, others not). The translation node builds on top of versioned messages. In that sense, the translation supporting the versioning sounds backwards to me. How do you see this?

[hamishwillee]

I haven't looked at the example, but it seems to me that this might require some worked examples of each type. If the headers are specific to each type, perhaps move them under the following sections?

[GuillaumeLaine]

You're right, I've gone and added a worked example also to make the steps clearer

[hamishwillee]

So what versions will it work for?

[bkueng]

Jazzy or later

[hamishwillee]

@GuillaumeLaine So ROS Jazzy is a dependency - if this is never going to be in ROS Humble we should maybe say

Suggested change

	- The current implementation depends on a service API that is not yet available in ROS Humble, and therefore does not support services when built for ROS Humble.
	- The current implementation depends on a service API that is present from ROS Jazzy (ROS applications build for Humble and earlier cannot be used with the node).

I added a version for ROS Jazzy above on this doc. We should also add it in the guide, as noted in https://github.com/PX4/PX4-user_guide/pull/3465/files#r1924601823

[hamishwillee]

Do you mean services as in something different than message/topics? Perhaps this is what you mean by .svr above? Is there information on what those mean anywhere?

Either way, so are you saying that services require a linear history, but you can split or merge messages as you like?

[GuillaumeLaine]

Correct, ROS distinguishes between services and topics like so: https://docs.ros.org/en/foxy/How-To-Guides/Topics-Services-Actions.html. In PX4, we have the srv/ services folder vs the msg/ topics folder.

Messages define the data format used to communicate over either topics or via services. The confusing part is that .srv files define service messages, and .msg files define topic messages (the file extension introduces confusion and make them seem more generic)

And finally yes, currently services require a linear history to translate while topics (.msg) can be split and merged

Thanks for the comment, I realized I was also confused by this too. I'll do my best to make this clear in the docs

[hamishwillee]

So a ROS application must compile against just one version of a topic? What if there are two ROS applications running on the network compiled against different versions - is this the problem case?

[bkueng]

So a ROS application must compile against just one version of a topic?

Yes

What if there are two ROS applications running on the network compiled against different versions - is this the problem case?

No, only a specific subset, like:

• app 1: pub topic_v1, sub topic_v1
• app 2: pub topic_v2, sub topic_v2

In practice there's rarely (if any) setup that will run into this.

[hamishwillee]

I don't quite get what is being said here. I think you're saying this is OK:

app 1: pub topic_v1, sub topic_v1
app 2: pub topic_v2, sub topic_v2

But this is not

app 1: pub topic_v1, sub topic_v2

Right?

[bkueng]

Not quite, it needs both apps with these topics to be a problem. It's really quite specific.

[hamishwillee]

@GuillaumeLaine

OK, lots of comments here. They don't reflect that this is pretty good structure. Just I'm using myself as the straw man, and I don't know much ROS 2.

When you've had a look at the comments and tried address them, can you loop in Beat and Benja to have a look? I ask because I probably will be on holiday when you do this. Back around week of Jan 20th.

[mrpollo]

Maybe worth adding to the 1.16 release wip page? @hamishwillee

[bkueng]

Nice work, thanks for going through this

[bkueng]

Jazzy or later

[bkueng]

So a ROS application must compile against just one version of a topic?

Yes

What if there are two ROS applications running on the network compiled against different versions - is this the problem case?

No, only a specific subset, like:

• app 1: pub topic_v1, sub topic_v1
• app 2: pub topic_v2, sub topic_v2

In practice there's rarely (if any) setup that will run into this.

[GuillaumeLaine]

@hamishwillee Thanks for the thorough review! I've expanded on what was there, hopefully this addresses most of your comments. Let me know if parts are still unclear when you're back.

In the meantime @beniaminopozzan would you like to have a look? Hamish told me might be interested in being kept in the loop

[beniaminopozzan]

Thanks @GuillaumeLaine !
I just pointed out a few typo. Nothing else for me to report - just exited to see it finished!

[hamishwillee]

I modified this line. Please check it is correct.

FWIW I think the updates to this doc work well.

[hamishwillee]

I've subedited this a little, mainly "to my taste".

It is still a problem for me that this is the first and only mention of "topic message files and service message files" and that there is no definition of what these are for, or where they are from. If you're from ROS-land perhaps you know, but I can assure you that many readers will not.

Can you create a section in this topic which outlines the differences/similarities, and when you should create one or the other. Can be a different PR.

[hamishwillee]

I understand this diagram, but it took me a little while to understand what it meant - specifically what versions are supported. Do you think this makes it easier?

[hamishwillee]

This is all quite hard to describe because the terminology is messy - the versioned message is both the new and old message at different points. I have tried to make this more clear by defining versioned message as "the file in this location". Similarly tried to do the same with the step points.

Not sure if this was strictly necessary, because your examples in each step make things very clear.

[hamishwillee]

It would be good to have an example of a generic translation and a service translation. This could be minimal, or a pointer to an example in the codebase once one exists.

[github-actions]

No flaws found