Integration of docstr_coverage into Sphinx

[Paebbels]

Hi,

I spent some time writing a quick and dirty Sphinx extension, so outputs from docstr_coverage are listed in Sphinx. The result for 1 day of coding and debugging looks as follows:

Are you interested in such a usage of your package.
Could you support in improving it?

[MiWeiss]

Wow, this looks amazing :-) I really hope that you make this into an openly available, public project.

How are you using docstr-coverage in you plugin? I hope you use the analyze function, which is intended pretty much exactly for such purposes.

from docstr_coverage import analyze
coverage_report = analyze(['some_dir/file_0.py', 'some_dir/file_1.py'])

Your questions

Are you interested in such a usage of your package.

Absolutely, it looks amazing.

Could you support in improving it?

At the moment, not too much development is going on on our side, due to the lack of time and as this project is comparably stable. I likewise doubt we'll find much time committing to your source.

However, I am fairly certain we will be able to provide you with the following:

• Answers to any questions you may have about the internals of this package
• Quick reviews and merging of any small PRs you'd be submitting to this codebase
• Promotion of your plugin once it is publicly availabe and ready for usage in our README (i.e., containing the most important features, and having those documented, tested)

Note that @HunterMcGushion has the last say on everything, but I am quite confident that he'll agree.

[Paebbels]

Yes, I use analyze(...).

Your answers

Are you interested in such a usage of your package.

Absolutely, it looks amazing.

My plan is a new package at pyTooling/sphinx-reports for the integration of these report types:

• documentation coverage using docstr_coverage (and interrogate?)
• code/branch coverage reports by coverage.py
• test summary reports by pytest
• mypy reports (far future)

This would be Apache License 2.0 code, which is compatible with MIT.

Could you support in improving it?

At the moment, not too much development is going on on our side, due to the lack of time and as this project is comparably stable. I likewise doubt we'll find much time committing to your source.

I'm active in lots of Python code, but I would like to spend my time not so much in sphinx extensions. It is already a painful experience writing Sphinx extensions, because Sphinx as a documentation tool itself is the by far the worst documented tool I've ever seen. Everything in this tool is based on magic and side effects.

Anyhow, I would like to bring this to a stable release but then it would be good if others could help maintain it. Maybe from the docstr_coverage community?

Besides my real job, I'm writing a new Sphinx domain (at least I try to write one) for VHDL. With this idea in mind, lots of side projects evolved like a VHDL parser written in Python, a GHDL to Python lolevel C binding, a 9-layered EDA² framework as well as pyTooling and pyTooling/Actions. As I'm working on v6.0.0 of pyTooling, I'm also rewriting major parts of the documentation and theme styling (as far as I understand CSS) - which broad me to your package.

---

Features I have so far:

• Multiple directory lists identified by an id can be configured in conf.py. This allows to have separate coverage reports for a src, tests or example directory.
• Every coverage table row - listing one Python module - refers to a CSS class doccov-aboveXX, where XX is 00 to 90. This allows the definition of color coding in 10% increments. CSS styles can be grouped to apply same colors to multiple classes.
• All output is based on docutils, thus the table is a native table and it should translate to any backend format (builder) supported by Sphinx.

Problems I'm facing so far:

• How to find a good CSS styling - I'm no HTML/CSS/web programmer.
• How to connect my own CSS file with the extensions. Currently I'm using a global styles overwrite file. I need to debug existing extensions how they solve that problem.
• I need to shorten the paths for every Python module file, so the relative path from ./doc to ../src isn't visible in the table.

What I noticed so far for docstr_coverage:

• There is a minor mismatch in found, covered and uncovered elements compared to interrogate (±3).
  At least, both packages find 3x more elements then sphinx.ext.coverage. This Sphinx builder is not usable as a plugin nor does it compute the coverage in a fast way. It compiles a full documentation first, to emit a coverage.
• When executing analyze(...), the progress bar is printed to the console.
  For now I'm not sure if this is a good or bad behavior. Normally it's bad when a plugin has direct console/print access. The it can't be used as a lower-layer library.
• The packages misses many type hints, so I needed to type hint / or cast in my code so I could use the package. We should improve that.
• Some objects need a more extensive API to be easily usable.

What I have in mind:

• I created an abstract coverage data model for Python code coverage.
  Normally, I provide such an initial data model as an independent package at GitHub/PyPI (and sare access). Then frameworks use and implement such a data model (here docstr_coverage and/or interrogate?). So your package becomes a consumer of the data model layer. Here you add your content by deriving the classes and implementing abstract methods or by creating instances of the model. Finally, the report extension package would again consume your package.
  This is how EDA² - a 9 layered stack is designed. Other data sources could create such a data model instance but mor interestingly for you, more higher-level packages can consume your data.
  As this layer is crucial to your package, I would share access on the GitHub repository and/or the GitHub organisation pyTooling.
• Alternatively, I would write a converter from your data model to my data model.
  This is a lot of translation code and maintenance that I dislike. I would like to avoid it upfront.
• I hope someone of your community - this should be skilled Sphinx users and maybe some (extension) programmers could help a bit on that package.

[HunterMcGushion]

Hi, @Paebbels ! This is really cool! As @MiWeiss said, we aren't doing too much active development on docstr-coverage currently for the reasons he already mentioned. Likewise, I don't think either of us (please correct me if I'm wrong, @MiWeiss ) can make any substantial time commitment to help develop pyTooling and any extensions to communicate with docstr-coverage. You make a lot of excellent points, especially about the verbose logging in this particular case.

How to connect my own CSS file with the extensions. Currently I'm using a global styles overwrite file. I need to debug existing extensions how they solve that problem.

This may not be exactly what you're looking for, but I did some minimal styling for the HyperparameterHunter docs by adding docs/_static/style.css and then loading it in docs/conf.py

How to find a good CSS styling - I'm no HTML/CSS/web programmer.

I'm certainly not a frontend engineer either, but I'd recommend looking into things like Bootstrap or MaterialUI, which provide large libraries of styles and components. I don't know how usable either of these will be if you're limited to a single CSS file, but I think they're good starting points to land on some basic UI that looks presentable.

Again, your project looks really awesome, and I'm honored that you want to add support for docstr-coverage to pyTooling! Thank you for sharing it with us! I'd love to continue the discussion here if you have any other questions.

[Paebbels]

The repository is now public: https://github.com/pyTooling/sphinx-reports (see dev branch) or PR.

I'm currently migrating from a local extension to a real extension. Then I would like to change it from a simple extension to a domain.

[Paebbels]

Update

So I just release v0.3.1, which should be a usable version for first tests by other users / early adopters. It fixes many remaining bugs and also ships the CSS file as a package resource.
Please have a try.

Repository: https://github.com/pyTooling/sphinx-reports
Extension setup instructions:https://pytooling.github.io/sphinx-reports/Overview.html
Doc coverage explanations: https://pytooling.github.io/sphinx-reports/DocCov/index.html

The documentation also contains 3 examples with no, some and a full documentation.

Moreover, the extension is now able to fully recreate the hierarchy to display packages and modules as a tree.

Questions

1. Do you have suggestions on coverage levels?
   I'm currently using <30, <50, <80, <90, <100%.
2. Can you provide more detailed coverage data then on a file basis?
   My data model also supports classes, methods, fields, functions, etc.
   See data model here: https://github.com/pyTooling/sphinx-reports/blob/main/sphinx_reports/DataModel/DocumentationCoverage.py