ar_tools

Uses:

• halodi_msgs
• OpenCV 4.4.0 + contrib
• numpy and scipy

Build: colcon build --packages-select ar_tools

This package publishes TFs for Aruco markers from cameras, currently supporting Zed (left eye) and image streams over gRPC. Nodes also publish transformed poses in halodi_msgs/msg/ARMarkers form.

Running aruco_ nodes requires a config file; refer to config/aruco.json for an example.

aruco.json

• parent_frame: frame to transform markers into
• camera_frame: frame that markers are detected in
• broadcast_transforms: if true, publishes camera -> marker TF msgs
• image_scaling: fx, fy of cv.resize
• aruco_dict: identifying string that follows OpenCV's pre-defined dictionaries, omitting DICT_
• marker_sizes: dictionary of numerical marker IDs to side lengths, used for pose estimation. Markers not defined here are discarded
• aruco_params: public fields for a parameters object

AR marker streaming nodes

stf_server

ros2 run ar_tools stf_server FREQUENCY

Convenience node to fetch stamped transforms for timestamps which are not associated with a clock topic. Used by all aruco_ nodes via halodi_msgs/srv/GetStampedTF. Due to use of Python's time.perf_counter() for internal time-keeping, calls to this service should only be made from the machine running the service server. For messages on /tf_static, it is assumed that non-zero timestamps represent an additional sensor delay to the timestamp in transform requests.

• FREQUENCY: rate throttling for subscribers, to help reduce CPU load and network traffic.

aruco_zed

ros2 run ar_tools aruco_zed PATH_TO_ARUCO_CONFIG_FILE STREAMING_ADDRESS

• STREAMING_ADDRESS: address of a stream to listen to, e.g. 10.0.0.1:30000. If no colon is detected in this argument, a non-network connection is assumed.

aruco_grpc

ros2 run ar_tools aruco_grpc PATH_TO_ARUCO_CONFIG_FILE GRPC_IMAGE_SERVER GRPC_COMMON_SERVER

• GRPC_IMAGE_SERVER: address of a GRPC image server, e.g. 10.0.0.1:30000. Data structure follows single-image syntax of azure_grpc
• GRPC_COMMON_SERVER: address of a GRPC common server

Extrinsic calibration nodes

Running calibration_extrinsic_ nodes requires a config file; refer to config/extrinsic_calibration.json for an example. Requires a marker fixed in space e.g. on a wall, visible to the robot across a reasonable range of motion with a streaming node already running, e.g. aruco_zed. Robot is assumed to be moving while data is being collected, e.g. through a calibration_motion_ node. Note that the first element of the parameter vector to be optimized is assumed to be a camera delay (>=0).

extrinsic_calibration.json

• common: contains parameters used by all extrinsic calibration nodes
  • tf_bookend_duration: additional subscription time to TF before and after subscribing to marker msgs
  • data_collection_duration: time for subscribing to both TF and marker msgs
  • data_collection_samples_n: reduce total marker samples to >= this value after collection
  • markers_topic: topic for markers
  • stationary_target_frame: frame ID for a marker that is stationary relative to the robot
  • static_frame: a fixed TF frame ID relative to the robot, nominally the root of the TF tree
  • camera_frame_parent: the immediate parent frame ID of the camera
  • camera_frame: frame ID of the camera
  • outbound_calibration_topic: for publishing calibration info if optimization is successful
  • camera_name: camera name for publishing
  • de: some args for differential evolution
  • project_rotations: whether or not the projection of a unit vector from the target's estimated quaternion should be taken as an additional criterion
  • verbose_optimization: prints standard deviation vector in each call to the error fn
  • data_save_folder: folder to write collected data to. No writing is performed if this string is empty
• head: parameters for calibrating a camera mounted in/on the head
  • mean: [ camera_delay, head_pitch_offset, head_to_camera_xyz_ypr ]
  • extents: range of mean. DE bounds are calculated by mean +/- extents

calibration_extrinsic_head

ros2 run ar_tools calibration_extrinsic_head PATH_TO_CALIB_CONFIG_FILE CMD

For calibrating a camera mounted in or on the head (only movement of the camera relative to the robot's kinematic root is through the neck joint).

• CMD: a command sequence, eg "collect_optimize_publish". Either "collect" or "load" must be present
  • collect: to collect new data
  • load: to load from "data_save_folder" in extrinsic_calibration.json
  • optimize: run optimization
  • publish: publish extrinsic calibration info after publish

calibration_motion_head

ros2 run ar_tools calibration_motion_head

Runs a looping routine that moves the neck and pelvis.

Launch files

Please adjust arguments in relevant launch/config files, then cd to the workspace root and run "source install/setup.bash" prior to use.

• ros2 launch ar_tools grpc.launch.py: aruco_grpc + stf_server
• ros2 launch ar_tools zed.launch.py: aruco_zed + stf_server
• ros2 launch ar_tools extrinsic_calibration_head.launch.py: calibration_motion_head + calibration_extrinsic_head (required)