Gazebo | Ignition | Community
Ask Your Question
0

How to start multiple Gazebo simulations with own rosmasters

asked 2019-09-25 12:54:18 -0600

Tima15 gravatar image

Hello!

I am just trying to train my algorithm to navigate without colliding with obstacles. Here I am using a RL algo that is trained with the corresponding gazebo world. The problem is now, that with highest real time update rate, one simulation with 5000 episodes needs about 25h. I now checked out htop and the performance or the graphics is always concentrated on one core. That why I am now trying to find out, how to get multiple simulations running in parallel. My goal is then to better use the capacity of my PC to have more simulations done in the same time. I first checked out this and this page to be able to get more than one gazebo simulation and rosmaster running in parallel.

Here comes my first question. I can copy the commands from those websites to get multiple simulations and rosmasters. But how can I tell one rosmaster, that he has to communicate with a certain gazebo node (especially, if the /gazebo messages can not be changed and are thus the same for the simulations)?

My second question is about the multimaster_fkie package. I am quiet new to that one and I found this page. There my goal is achieved of having seperate simulations. Can this package be used to tell a rosmaster to communicate only with one gazebo simualtion so that I can have more simulations in total with different rosmasters?

Best regards

edit retag flag offensive close merge delete

2 Answers

Sort by » oldest newest most voted
0

answered 2019-10-04 04:03:01 -0600

Tima15 gravatar image

Okay, the first answer is correct despite of two small things:

  1. for the second tab it must be rosrun fkie_master_discovery master_discovery _mcast_group:=224.0.0.251. I think the documentary is maybe a bit old.
  2. For the second tab in the second terminal it should be rosrun fkie_master_discovery master_discovery _mcast_group:=224.0.0.252 _mcast_port:=11512 to prevent same adress compared to the first terminal
edit flag offensive delete link more
0

answered 2019-09-26 03:41:47 -0600

kumpakri gravatar image
Can this package be used to tell a rosmaster to communicate only with one gazebo simualtion so that I can have more simulations in total with different rosmasters?

Yes.

The Multimaster-FKIE package connects the different ROS masters, checks for the activity on the master and copies all the messages it finds to all of the connected ROS Masters (connects nodes of the different ROS masters based on the topics they subscribe or publish to). The Multimaster-FKIE package provides an ignore list where you can specify the topics or the nodes that are not to be connected with other ROS masters.

By default the [/rosout, "/master_sync node", "remote /master_discovery node", "/node_manager", /zeroconf] nodes and the ['/rosout', '/rosout_agg'] topics are ignored. But you can add your own on top of that.

I have successfully run two simulations on two different ROS masters by adding the ['/gazebo','/tf','/clock'] topics into the ~ignore_topics list.

Here are some notes I wrote for myself when exploring the Multimaster-FKIE package:

Running Multiple Simulations Using Multiple ROS-Masters

multimaster-FKIE package

The multimaster-FKIE package provides two main nodes

  • master_discovery
  • master_sync

The master_discovery node broadcasts the messages over the network and scans the network for broadcasts of other master_discovery nodes. It then publishes the communication of other master_discovery nodes to its own ROS-Master. The master_sync connects to the discovered master_discovery nodes and requests the actual ROS state and registers the topics and services of the other ROS-Master.

It is possible to set whether to synchronize or ingore for each node, topic or service. This way certain topics or nodes can be private for the ROS-Master and will not be communicated to the other ROS-Masters.

image descriptionsource:ros.wiki.org

Simulation A and B example

There are two ROS-Masters running at different ports (11311 and 11312). Both ROS-Masters run a master_discovery node and a master_sync node. The /gazebo, /gazebo_gui, /tf and clock topics are set private for each ROS-Master.

The Gazebo simulation A is launched at the port 11345 and connected to the ROS-Master at the port 11311. The Gazebo simulation B is launched at the port 11346 and connected to the ROS-Master at the port 11312.

How to install

install multimaster-fkie

sudo apt-get install ros-kinetic-multimaster-fkie

check if multicast allowed

cat /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts

if output 0, multicast is allowed

allow multicast temporarily

sudo sh -c "echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts"

allow multicast permanently

edit the /etc/sysctl.conf and add following line

net.ipv4.icmp_echo_ignore_broadcasts=0

and execute command

sudo service procps restart

Check which multicast groups are defined in a computer

netstat -g

Running The Simulations A and B

First terminal

tab 1

roscore

tab 2

find the group

netstat -g

netstat

run the master_discovery

rosrun master_discovery_fkie master_discovery _mcast_group:=224.0.0.251

tab 3

run master_sync

rosrun master_sync_fkie master_sync _ignore_nodes:=['/gazebo*','/tf*','/clock'] _ignore_topics:=['/gazebo*','/tf*','/clock']

Here the ignore nodes might be redundant but I didn't want to take chances. You can experiment.

tab 4 and more

source ~/catkin_ws/devel/setup.bash

Run Simulation ... (more)

edit flag offensive delete link more

Comments

Thanks for the answer! But first thing that occurs is, that the master_discovery_fkie package does not exist. Do you mean the fkie_master_discovery package (tab 2 and tab 3 for sync as well)? Furthermore, when I try the second terminal, the tab 1 gives me the following warning: WARNING: ROS_MASTER_URI port [11311] does not match this roscore [11312] Continued by the tab 2 command error: Unable to bind multicast to interface: 224.0.0.251, check that it exists: [Errno 98] Address already in use

Tima15 gravatar imageTima15 ( 2019-09-26 06:18:22 -0600 )edit

Also, when i am doing the rosservice call /master_discovery/list_masters command, I can only see the first master.

Tima15 gravatar imageTima15 ( 2019-09-26 06:27:19 -0600 )edit

According to this documentation the `master_discovery_fkie' is the correct name.

The WARNING: ROS_MASTER_URI port [11311] does not match this roscore [11312] warning is correct and you don't need to worry about it.

the 224.0.0.251 network group is on my computer. You can see what your network group is called using the netstat -g comand and replace my group address for yours in the _mcast_group parameter.

kumpakri gravatar imagekumpakri ( 2019-09-26 08:19:55 -0600 )edit

The rosservice call /master_discovery/list_masters command will only show the ROS masters already connected through the network.

kumpakri gravatar imagekumpakri ( 2019-09-26 08:20:37 -0600 )edit

Thanks again for the answer! I did the netstat -g and got exactly the same number (224.0.0.251). That´s why I did not change that number. I tried it again and got: Create multicast socket at ('224.0.0.251', 11511): Unable to ... again.

Still, when I do rosrun master_discovery_fkie master_discovery _mcast_group:=224.0.0.251, the Error: package 'master_discovery_fkie' not found occurs (same for master_sync_fkie).

Tima15 gravatar imageTima15 ( 2019-09-26 08:53:38 -0600 )edit

Did you allow the multicast on your computer as described in the instructions above?

kumpakri gravatar imagekumpakri ( 2019-09-26 09:14:56 -0600 )edit

is the roscore running on the corresponding port?

kumpakri gravatar imagekumpakri ( 2019-09-26 09:21:36 -0600 )edit

Yes, i did allow the multicast (the 0 appears in the terminal with cat /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts command). For the port, I now tried roscore -p 11312 and roscore --port=11312 for setting it and then export ROS_MASTER_URI=http://localhost:11312 in the second tab. But then the error orrcurs again:

Create multicast socket at ('224.0.0.251', 11511): Unable to bind multicast to interface: 224.0.0.251, check that it exists: [Errno 98] Address already in use.
Tima15 gravatar imageTima15 ( 2019-09-26 10:30:39 -0600 )edit

I guess there is probably something running on your computer that blocks the port. I don't have any experience with that, so you need to find a way how to check which process it is and how to kill it.

kumpakri gravatar imagekumpakri ( 2019-09-27 04:16:50 -0600 )edit

Question Tools

2 followers

Stats

Asked: 2019-09-25 12:54:18 -0600

Seen: 11,011 times

Last updated: Oct 04 '19