Reconsider documentation style for private symbols

Background

We currently do not document API private classes and functions using the standard docstrings, but in a special comment-based style. This means that they are unusable by IDEs or even the standard help() in a REPL (both of which I find quite useful personally).

Since Sphinx already does not include private symbols by default, I'm not sure if this buys us anything. Their documentation says that that the members option:

will document all non-private member functions and properties (that is, those whose name doesn’t start with _).

Isn't this exactly what we want? Or am I missing something obvious here?

I noticed this most recently in !1918 (merged), where there are proper docstrings for functions like buildstream.utils._make_protobuf_timestamp, and they do not appear in the output. You can preview the output at: https://buildstream.gitlab.io/-/buildstream/-/jobs/568430551/artifacts/public/buildstream.utils.html.

In my opinion, standard docstrings would help with the productivity of folks hacking on BuildStream, and I don't think we should underestimate the importance of that.

Curious to hear what others think about this.

Possible solution

I propose to phase out this comment style documentation in favor of standard docstrings. On a related note, I also think we can remove this custom logic from our makefile, since that's what Sphinx does by default anyway.

Implementation details

If we do decide that it's a good idea to do this, then there's also the matter of how to do the transition.

If we can automate this somehow, then ideally we'd run this as a one-off script. Everyone with exisiting branches will have to bite the bullet once, and do some potentially tedious rebases. But, if the script is idempotent then people could just run that script on their branches first, which would makes things much easier.

If the automated options seems unpractical, or isn't ready yet, I would propose a more incremental approach. A policy like "clean-up what you touch" might help do this gradually.