Gazebo | Ignition | Community
Ask Your Question

Using TensorFlow with Gazebo

asked 2016-11-03 16:00:19 -0600

CyborgOctopus gravatar image

Hi, I'm new to computer programming and this simulator. I'm trying to do a machine learning project, specifically creating learning robots with neural networks. It seems like plugins use C++, so is there any way to control simulated robots in Gazebo with TensorFlow? I have heard about an extension to the OpenAI gym that uses Gazebo, but it only has a few pre-defined robots and environments.

edit retag flag offensive close merge delete


holy shit, that's a thing that exists?!!?

Peter Mitrano gravatar imagePeter Mitrano ( 2016-11-03 18:18:12 -0600 )edit

2 Answers

Sort by ยป oldest newest most voted

answered 2016-11-03 18:24:32 -0600

updated 2016-11-04 15:37:32 -0600

So for starters, I presume you know that tensorflow is really best done in Python, and the C++ api seems like it only lets you execute graphs, not train them. However, you can use ROS to talk to Gazebo, and ROS supports python. So yes, you could build a robot or use on of the few existing ones (turtlebot would be the best to start with), write some ROS code that does some tensorflow vodoo and publishes commands to the robot.

some gazebo/ros tutorials: guided tutorial

generic 1 2 3

edit flag offensive delete link more


by the way, this is a super awesome project and you should post about it somewhere if you make progress. a hell of a lot of awesome tech to put together

Peter Mitrano gravatar imagePeter Mitrano ( 2016-11-03 18:26:31 -0600 )edit

Thanks for the answer! It's really helpful. Is there a specific tutorial that explains how to publish ROS commands to a Gazebo robot? Also, I definitely will post something about my project once I get it finished.

CyborgOctopus gravatar imageCyborgOctopus ( 2016-11-04 07:32:24 -0600 )edit

If you're using turtlebot you won't have to worry about gazebo, the ROS stuff is super well documented and thousands of people use it as a learning platform for ROS. However, I've updated my answer with some links to tutorials

Peter Mitrano gravatar imagePeter Mitrano ( 2016-11-04 15:35:40 -0600 )edit

Okay, thanks.

CyborgOctopus gravatar imageCyborgOctopus ( 2016-11-04 20:32:54 -0600 )edit

answered 2017-01-27 06:56:58 -0600

peci1 gravatar image

I agree the "ROS way" would be the easiest one.

I tried using the C++ API of tensorflow in Gazebo, and there you are stopped by the fact that Tensorflow requires libprotobuf >= 3, but Gazebo requires libprotobuf < 3. This is generally a big problem if you want to link your app. I'm right now trying some tricks with PIMPL and static linking, so let's see if it's at least possible with a lot of effort.

edit flag offensive delete link more

Question Tools

1 follower


Asked: 2016-11-03 15:53:15 -0600

Seen: 3,622 times

Last updated: Jan 27 '17