Synchronous slam and localization/navigation not working

[schenkch]

Please provide the following information:

• OS: Ubuntu 20.04
• ROS Distro: Galactic with CycloneDDS
• Built from source or installed: installed

Expected behaviour
When starting localization or navigation, I expect all the sensors/transforms to be published in rviz after waiting some time, such that I can actually run navigation by setting a start and goal.

Actual behaviour
I can run asynchronous slam via ros2 launch turtlebot4_navigation slam_async.launch.py but not synchronous slam. Then in rviz the transforms are not published and it looks like this:

The same when trying to run localization or navigation with and without slam via ros2 launch turtlebot4_navigation nav_bringup.launch.py slam:=off localization:=true map:=office.yaml

The logs from rviz show an error that I have never seen before:

[INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2023-07-06-12-55-22-619089-Cubi02-30924
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [rviz2-1]: process started with pid [30926]
[rviz2-1] [INFO] [1688640924.066465443] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-1] [INFO] [1688640924.066792348] [rviz2]: OpenGl version: 4.6 (GLSL 4.6)
[rviz2-1] [INFO] [1688640924.147039648] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-1] [WARN] [1688640924.683929490] [rcl.logging_rosout]: Publisher already registered for provided node name. If this is due to multiple nodes with the same name then all logs for that logger name will go out over the existing publisher. As soon as any node with that name is destructed it will unregister the publisher, preventing any further logs for that name from being published on the rosout topic.
[rviz2-1] [WARN] [1688640924.703287420] [rcl.logging_rosout]: Publisher already registered for provided node name. If this is due to multiple nodes with the same name then all logs for that logger name will go out over the existing publisher. As soon as any node with that name is destructed it will unregister the publisher, preventing any further logs for that name from being published on the rosout topic.
[rviz2-1] [WARN] [1688640924.725581661] [rcl.logging_rosout]: Publisher already registered for provided node name. If this is due to multiple nodes with the same name then all logs for that logger name will go out over the existing publisher. As soon as any node with that name is destructed it will unregister the publisher, preventing any further logs for that name from being published on the rosout topic.
[rviz2-1] [INFO] [1688640925.121231423] [rviz2]: Trying to create a map of size 60 x 66 using 1 swatches
[rviz2-1] [ERROR] [1688640925.150834879] [rviz2]: Vertex Program:rviz/glsl120/indexed_8bit_image.vert Fragment Program:rviz/glsl120/indexed_8bit_image.frag GLSL link result :
[rviz2-1] active samplers with a different type refer to the same texture image unit
[rviz2-1] Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[rviz2-1] at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.5/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[rviz2-1] at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.5/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame does not exist
[rviz2-1] at line 156 in /tmp/binarydeb/ros-galactic-tf2-0.17.5/src/buffer_core.cpp
[rviz2-1] Warning: Invalid frame ID "map" passed to canTransform argument target_frame - frame d

To Reproduce
I reinstalled all ros2 packages and turtlebot4-desktop via
sudo apt update && sudo apt install ros-galactic-turtlebot4-desktop
and turtlebot navigation via
sudo apt install ros-galactic-turtlebot4-navigation

Any ideas on how to fix this are highly appreciated?

[hilary-luo]

Can you verify that you are not accidentally running nav2 / localization from two places at the same time? (e.g. on the robot and on your pc).

Assuming that that is not the case can you please provide the terminal output from when you launch synchronous slam?

[schenkch]

Yes, that is not the case. Here are the logs of running synchronous slam and rviz.
logslam.txt
logrviz_slam.txt

And here of running localization without slam:
logrviz_z_localization.txt
log_localization.txt

There seems to be a time synchronization issue as well. So, I tried restarting the ntpd on the create3 web platform but that has not solved the problem. However, I definitely do not know what this error:
[rviz2-1] [ERROR] [1688640925.150834879] [rviz2]: Vertex Program:rviz/glsl120/indexed_8bit_image.vert Fragment Program:rviz/glsl120/indexed_8bit_image.frag
is coming from. Any advice?

[hilary-luo]

I am not familiar with that last error that you mentioned. That being said, the core error seems to be this one:

[controller_server-5] [INFO] [1688714280.030831185] [local_costmap.local_costmap]: Timed out waiting for transform from base_link to odom to become available, tf error: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist

I believe that the timestamp errors are a result of this timeout.

Are you able to see the Create3 topics? Ensure that your networking is setup properly and that you can see all of the topics from the raspberry pi and the create 3 base. If you are using Simple Discovery then you can try using Discovery Server and route all of the data through the pi.

Additionally, can you try and run slam on the raspberry pi itself? This will confirm that the issue is in the networking / remote pc.

[schenkch]

Yes, I can see all the Create3 topics from the RPi and remote PC. I just tried it with the RPi and remote PC again. And for running synchronous slam I can see all transforms in rviz as well now on both machines but I still get this error:

[rviz2-1] [ERROR] [1688640925.150834879] [rviz2]: Vertex Program:rviz/glsl120/indexed_8bit_image.vert Fragment Program:rviz/glsl120/indexed_8bit_image.frag

For running navigation/localization the transforms still do not show up in rviz.

[hilary-luo]

Can you please run navigation / localization on the raspberry pi itself and let me know the results?

That RVIZ Vertex error seems to have to do with the shading of objects in rviz. Does your rviz show any elements that are pure white?

[schenkch]

Yes, it is not working either. Please find the logs here:
logrviz_navasyncrpi.txt
lognav_asyncrpi.txt

Do you mean pure white elements in the map like here?

I will try to switch to discovery server again. Maybe I can get it working now after changing my WIFI network.

[schenkch]

I switched to fast discovery and I thought that I got the communication working. It shows all the ros2 topics on both machines and can receive /tf /scan /odom etc. data but I cannot control it with the controller and when I run asynchronous slam and rviz, it does not show anything. It still seems to be the time synchronization issue. I tried what is described here #62 and also installing sudo apt install ros-galactic-joint-state-publisher and then sudo apt install ros-galactic-joint-state-publisher.

Also I tried using the ntpd restart feature from the webserver and restarting the turtlebot4 but so far without success.
It does not work from the rpi nor from the remote PC.

Here are the logs from running asynchronous slam and rviz on the PC, the logs from the Create3 and what it looks like when I open rviz:

[log_async_dd.txt](https
logrviz_async_dd.txt
://github.com/turtlebot/turtlebot4/files/12038239/log_async_dd.txt)
messages(1).txt

@hilary-luo Do you know how to fix this?

[hilary-luo]

Thanks for the detail in your responses and for confirming that the issue is also present when running on the pi. I see two primary issues that jump out at me in the logs that you have shared.

1. The create3 base seems to be exhibiting the 100% CPU usage issue 100% CPU usage in Humble iRobotEducation/create3_docs#381
   Unfortunately I don't have a solution for this yet but it is actively being worked on. This alone may be causing a lot if not all of the issues.

2. I am still concerned about this duplicate publisher message and the vertex error. What is your topic list before and after you launch slam? and when you are able to launch slam successfully, does the robot model show up correctly in rviz?

[schenkch]

@hilary-luo So, I moved back to simple discovery with CycloneDDS. When launching asynchronous slam the robot model shows up correctly apart from an error for the left and right wheel but I can fix this using this iRobotEducation/create3_sim#125.

Here are the topics before and after launching asynchronous slam successfully:
Before on the rpi:

Before on the PC:

During/after on the PC:

During/after on the rpi:

And here the rviz and slam logs again:
[logrviz_async.txt](https://gi
log_async.txt
thub.com/turtlebot/turtlebot4/files/12103617/logrviz_async.txt)

[schenkch]

Now it shows up in the same way when running synchronous slam as well but when running navigation it still does not show up correctly (see screenshot below or like I showed you above with the white areas). And it is definitely related to the 100% CPU issue. I just find it odd that at some point it was working but back then crashed during navigation.

Here are the topics before and after:
Before on the rpi:

Before on the PC:

During on the rpi:

During on the PC:

And here the logs:
logrviz_navasync.txt
log_nav_async.txt
And from the Create3:
messages(2).txt

[hilary-luo]

I will follow up here when we have some steps that you can take to address the high CPU usage. Let me know if there is anything else that needs to be addressed that you don't think is related to this issue.

[hilary-luo]

@schenkch Can you try the latest create3 firmware and see if it helps with the cpu usage? For Galactic that would be here: https://iroboteducation.github.io/create3_docs/releases/g_5_2/

[hilary-luo]

This issue is being closed due to inactivity. If you are still experiencing the issue, feel free to reopen this ticket when you are ready to continue the troubleshooting process.