Build speed : mkdocs vs Hugo

[jcyrss]

I have a medium size doc project containing about 200 md files.

When it was built with mkdocs-material. After several rounds, it generally takes about 3.20 seconds

mkdocs build -d _site
INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: D:\projects\byhy\hytest-projects\hytest-docs\_site
INFO    -  Doc file 'tut/faq/py/string.md' contains an absolute link '/tut/py/basic/09/#split', it was left as is. Did you mean
           '../../py/basic/09.md#split'?

INFO    -  Documentation built in 3.20 seconds

And with Hugo, it is a different project folder but with almost the same markdown content, it generally takes about 0.7 seconds.

Start building sites … 
hugo v0.122.0-b9a03bd59d5f71a529acb3e33f995e0ef332b3aa+extended windows/amd64 BuildDate=2024-01-26T15:54:24Z VendorInfo=gohugoio

                   | ZH
-------------------+------
  Pages            | 353
  Paginator pages  |  17
  Non-page files   |  11
  Static files     |   7
  Processed images |   1
  Aliases          |  26
  Sitemaps         |   1
  Cleaned          |   0

Total in 679 ms

It is testd on a HP laptop : EliteBook 835 13 inch G10 Notebook PC.

• AMD Ryzen 7 7840U
• 16GB 6400MHz RAM
• Micro SSD 3400 1TB M.2 SSD

Any comments about that?

[kamilkrzyskow]

On this page: https://gohugo.io/troubleshooting/performance/
they wrote that the pages are built in parallel and they have a cache system in-place for static components. On top of that Go is compiled.

MkDocs doesn't have that, the pages are processed sequentially one after another, and there is no caching of elements on the page like the header etc., perhaps only in part some theme templates that don't use the page object could be cached, but I doubt it 🤔
The more files you have to process the longer it will take. You could make a quick Copy and Paste to get 400 or 800 files and see how it will scale. Tough a file != another file, so perhaps use the ones that have more text inside them.

You can see the event flow in this diagram (you need to expand it): https://www.mkdocs.org/dev-guide/plugins/#events

As I wrote here you can use the --dirty flag when using mkdocs serve, and it will only rebuild files that have changed, resulting in a 0.5 seconds time.
You also didn't show the mkdocs.yml file, so I'm just assuming it's the default one with material theme enabled.

You could get the hook from this comment and see which stage of the build takes the most time, but it will be likely the same as I had in my own benchmarks.

The results are as expected when comparing Python and Go at scale.

[squidfunk]

If Hugo is built in parallel, I wonder whether features like tag listings or other things that depend on first collecting all page content before building are even possible to implement in Hugo. Parallelism was discussed over at MkDocs, but it's inherently impossible to implement on a plugin event level because it would take a lot of the power of the plugin system away.

Everything in software is a trade-off.

[jcyrss]

You could get the hook from #6781 (reply in thread) and see which stage of the build takes the most time

=============

I did that and got the following result:

INFO    -  Documentation built in 3.02 seconds
BUILD_STAGE_TIMES =
[('populate_pages', 2.02732),
 ('build_pages', 0.78763),
 ('build_theme_templates', 0.07296),
 ('get_files', 0.07033),
 ('copy_static_files', 0.02262),
 ('get_navigation', 0.0064),
 ('on_config', 1e-05),
 ('on_files', 1e-05),
 ('on_nav', 1e-05),
 ('on_pre_build', 0.0),
 ('on_env', 0.0),
 ('build_extra_templates', 0.0),
 ('on_post_build', 0.0)]
ALL_EVENT_TIMES =
[('page_markdown', 0.00542),
 ('page_content', 0.00011),
 ('page_context', 0.0001),
 ('post_page', 9e-05),
 ('pre_page', 6e-05),
 ('page_read_source', 4e-05),
 ('config', 0.0),
 ('pre_build', 0.0),
 ('files', 0.0),
 ('nav', 0.0),
 ('env', 0.0),
 ('pre_template', 0.0),
 ('template_context', 0.0),
 ('post_template', 0.0),
 ('post_build', 0.0),
 ('shutdown', 0.0)]
PLUGIN_TIMES =
[('_macros/hooks.py', 0.00201)]
PLUGIN_EVENT_TIMES =
[('page_markdown--_macros/hooks.py', 0.00201)]
sum(BUILD_STAGE_TIMES) = 2.9872900000000002
sum(ALL_EVENT_TIMES) = 0.0058200000000000005

[kamilkrzyskow]

Since the PLUGIN_TIMES doesn't include the search plugin, then there is no search index created. So this means that you have some custom mkdocs.yml file that removed the search plugin from the list, also no other plugin is visible apart of the macros hook 🤔

As mentioned previously in #6781 (reply in thread) the populate_pages step processes the Markdown into HTML, any markdown_extensions added to the mkdocs.yml process the Markdown text sequentially. The performance debugger hook doesn't include the times for each markdown extension, once I update it to support those then I will create a gist or something with the updated hook and I will report back here.
This step also includes the reading of the files, but even in parallel apps, accessing the file system is typically single-threaded.
Also I'm no Python-Markdown expert, so I don't know if recreation of the Markdown object for each page is necessary or wasteful. If it would be possible to re-use a Markdown processor for each page then it would increase performance to some extent.

The point still stands, that without parallelism or content caching the performance will degrade linearly to the amount of files/content to process.

For now if you want to continue checking the performance then start off with using a minimal project structure and build it up from there: minimal-with-performance-debug-hook.zip

EDIT: As a side-note, I heard that Python 3.13 will get some more performance optimizations on top of the ones in 3.11, so it should be worth it to check performance of the MkDocs again then.

[kamilkrzyskow]

I've created a gist with a bit better hook: https://gist.github.com/kamilkrzyskow/f8cd3e77aa8c297ac27b84c73fdbb767
It shows times for markdown processing and templates, maybe not perfectly, but should be still useful I hope. I admit, I haven't thoroughly tested the usefulness of it, just dabbled on the code bit by bit over the last few weeks and consider it "stable" for the current versions of overridden modules.

Here is the performance log for this repository: performance_debug.yml.txt
It's a serialized list of dicts, first entry is a summary with total times, second is the actual structure with the timings, and last is a structure with the count of Timer executions. I also disabled the Minify plugin before the test.

The main.html is timed 97 times, so 97 pages got processed and the build took 8.31 seconds. Out of that 5.48 seconds is the markdown conversion step, and this includes:

• 2.53 seconds for the markdown.treeprocessors.InlineProcessor,
• 1.91 seconds for markdown.extensions.toc.TocTreeprocessor,

so optimizing these builtin markdown processors would give the most gain in performance.
I don't know it's possible to optimize them (or hook a C++ solution instead of Python), just an observation.

Also I'm no Python-Markdown expert, so I don't know if recreation of the Markdown object for each page is necessary or wasteful. If it would be possible to re-use a Markdown processor for each page then it would increase performance to some extent.

This step only takes 0.078 seconds, so it's not wasteful, and I learned that processors require a reset of some kind to accept new content, so a complete recreation for each page resolved that.

The point still stands, that without parallelism or content caching the performance will degrade linearly to the amount of files/content to process.

This also now seems wrong, as times in POPULATED_PAGES and BUILT_PAGES aren't constant, but have a tendency do get shorter 🤔