Scene labeling using a depth camera
Is there any way to get a labeling of the scene as seen from a depth camera - a mapping between each pixel of the image and the corresponding object that generated the depth for that pixel as seen from the camera's viewpoint? So, for a scene with a single table in the middle, all the pixels corresponding to the table have to be labelled as "table". I've tried using the function:
GetVisualAt(CameraPtr camera, const math::Vector2i &mousePos)
from gazebo::rendering::Scene which casts a ray into the scene to find the intersecting object for a pixel but this is too slow for my use case (assuming a 640 x 480 image, this takes of the order of hundreds of seconds). I think that OGRE would compute this when it renders depth maps, but I'm not too sure how or where to look to get access to this. Any suggestions?
I'm trying to do something similar, it seems like there's a LogicalCamera class in the newer gazebo versions that seem to have some of this functionality (still not pixel-level segmentation). Any one know of how we could get pixel-level ground truth segmentation from gazebo?
Another similar question: http://answers.gazebosim.org/question/14422/is-anyone-aware-of-a-gazebo-simulated-depth-camera-that-identifies-the-object-each-depth-pixel-belongs-to/
Can you use "GetVisualAt" function successfully? When I'm using it, it complains: "gzserver: symbol lookup error: catkin_ws/devel/lib/librand_camera.so: undefined symbol: _ZN6gazebo9rendering6Camera6ZValueEii" Anyone knows the reason? I really appreciate it!