Using Gazebo without ROS?

edit

Using Gazebo without ROS?

Yes, Gazebo is intended to work as a standalone robot simulator / visualizer.

What I want to do is: Visualize a real robot in a 3D visualization together with additional stuff (like a "ghost" robot showing the goal config, little spheres indicating goal position, frames, etc). So basically I want a lightweight RViz without using ROS.

This may be possible via Gazebo Plugins in some of the later version of Gazebo.
See: http://answers.gazebosim.org/question...
and: https://ignitionrobotics.org/home

The documentation links back to ROS all the time so I don't really know if I can use Gazebo.

Have you looked through the Gazebo Tutorials? Many of them cover using Gazebo without ROS.

1) Can I simply load urdfs into the simulator (like with pybullet)?

Yes.
See: http://answers.gazebosim.org/question...

1a) If I can only load SDFs, is there a conversion tool? (Somehow when Gazebo is used in ROS, there are also only URDFs/Urdf-macros used.)

You can load URDFs directly into Gazebo without ROS, but you can also convert a URDF files into a SDF file, using
gz sdf -p my_urdf.urdf > my_sdf.sdf

2) Can I update robot joints continously in a program without ROS (and how?)

Yes, you can do this via a custom Gazebo Plugin.
See: http://gazebosim.org/tutorials?cat=gu...
and: http://gazebosim.org/tutorials?tut=se...
and: https://answers.ros.org/question/2330...

3) Can I disable all collision detection (basically the actual phys. simulation), so I can spawn robots into each other and don't have delayed actuation due to inertia?

Yes.
See: http://answers.gazebosim.org/question...
and: https://answers.ros.org/question/2187...