Robotics StackExchange | Archived questions

Using TensorFlow with Gazebo

Hi, I'm new to computer programming and this simulator. I'm trying to do a machine learning project, specifically creating learning robots with neural networks. It seems like plugins use C++, so is there any way to control simulated robots in Gazebo with TensorFlow? I have heard about an extension to the OpenAI gym that uses Gazebo, but it only has a few pre-defined robots and environments.

Asked by CyborgOctopus on 2016-11-03 15:53:15 UTC

Comments

holy shit, that's a thing that exists?!!?

Asked by Peter Mitrano on 2016-11-03 18:18:12 UTC

Answers

So for starters, I presume you know that tensorflow is really best done in Python, and the C++ api seems like it only lets you execute graphs, not train them. However, you can use ROS to talk to Gazebo, and ROS supports python. So yes, you could build a robot or use on of the few existing ones (turtlebot would be the best to start with), write some ROS code that does some tensorflow vodoo and publishes commands to the robot.

some gazebo/ros tutorials: guided tutorial

generic 1 2 3

Asked by Peter Mitrano on 2016-11-03 18:24:32 UTC

Comments

by the way, this is a super awesome project and you should post about it somewhere if you make progress. a hell of a lot of awesome tech to put together

Asked by Peter Mitrano on 2016-11-03 18:26:31 UTC

Thanks for the answer! It's really helpful. Is there a specific tutorial that explains how to publish ROS commands to a Gazebo robot? Also, I definitely will post something about my project once I get it finished.

Asked by CyborgOctopus on 2016-11-04 07:32:24 UTC

If you're using turtlebot you won't have to worry about gazebo, the ROS stuff is super well documented and thousands of people use it as a learning platform for ROS. However, I've updated my answer with some links to tutorials

Asked by Peter Mitrano on 2016-11-04 15:35:40 UTC

Okay, thanks.

Asked by CyborgOctopus on 2016-11-04 20:32:54 UTC

I agree the "ROS way" would be the easiest one.

I tried using the C++ API of tensorflow in Gazebo, and there you are stopped by the fact that Tensorflow requires libprotobuf >= 3, but Gazebo requires libprotobuf < 3. This is generally a big problem if you want to link your app. I'm right now trying some tricks with PIMPL and static linking, so let's see if it's at least possible with a lot of effort.

Asked by peci1 on 2017-01-27 07:56:58 UTC

Comments