I think I've found the answer. Thanks to chapulina for sending this link to the ODE user guide in the question comments.

The thesis of the below text is: **Most people will just want to use "mu" to set their coefficient of friction, and leave "mu2" blank.**

Anyways, if you go to section 3.11.1 in the userguide, it explains how they are implementing the ODE friction model. The model they use is an approximation of the friction cone, which is a well known model for friction. The cone only uses one coefficient of friction, which is what anybody who has taken physics is used to.

ODE uses a pyramid to approximate the friction cone to increase computational efficiency. The cone can be thought of as a polar coordinate system for the friction vector. Here the magnitude of the friction force would be NormalForce * FrictionCoefficient, and the angle would be opposite of the tangential force (or motion if no tangential force). The pyramid approximation used by ODE instead uses and x/y Cartesian coordinates to try and simplify the problem. This is what introduces the two "friction directions" and two friction coefficients. There is one friction direction and coefficient along the x direction of the square cross section of the pyramid, and one friction direction and coefficient along the y direction of the square cross section.

In section 7.3.7, they talk just a little about the two friction directions during a contact. It looks like you have the option to set the first friction direction in your SDF with the <fdir1> tag. If you don't set it, the documentation says the orientation of the first friction direction is unpredictable. The second friction direction is always calculated to be perpendicular to both the first friction direction and the normal vector. It also says in this section that if "mu2" is not explicitly set, it defaults to the same value as "mu".

In summary, it seems like "mu" and "mu2" apply two friction forces in unit vector directions that serve as components that make up a 2D friction vector. Why you would want two different coefficients of friction, even in this case, still remains unclear to me. It also remains unclear how "unpredictable" friction directions (if fdir1 is not set) can result in accurate friction forces.

I may try to mess around with "fdir1", "mu", and "mu2" in Gazebo to see how they actually work.

The Open Dynamics Engine manual has some extra clues: http://ode.org/ode-latest-userguide.html#sec_7_3_7

Thanks for sending that link. Their explanation isn't very good... I have a PhD in mechanical engineering and it still took me a while to understand what ODE is doing. I'll answer my own question based on what I read from that document... thanks!!!