Normalising Line-Endings

[href]

When \r\n is used for line endings, the behaviour can be a bit surprising:

>>> from mistletoe import Document, HTMLRenderer

>>> HTMLRenderer().render(Document("foo  \r\nbar"))
'<p>foo  \r\nbar</p>\n'

>>> HTMLRenderer().render(Document("foo  \nbar"))
'<p>foo<br />\nbar</p>\n'

I stumbled on this today and I thought I'd report it. Not sure what the correct behaviour should be though. I only found it mentioned here: https://talk.commonmark.org/t/carriage-returns-and-code-blocks/2519

[miyuchina]

I can't find any authoritative answers from CommonMark either. I used to replace \r\n line endings with \n in Document.__init__, but stopped doing so after I discovered that Python's open function automatically normalizes line endings. See the documentation on open's newline parameter.

Is there a reason you would want to pass in raw strings with CRLF line-endings to mistletoe? It is an easy fix--I just need to add one line in Document.__init__, but it will incur some performance cost where the normalization is otherwise not needed.

[JD557]

Is there a reason you would want to pass in raw strings with CRLF line-endings to mistletoe?

Just to put my issue in context, I noticed this in https://github.com/tinyBlogPub/microblog.pub.

When creating a post, the data from the textarea (which is then sent to mistletoe) seems to come with \r\n instead of just \n.

Unfortunately, I don't have enough insights to know who's the culprit here (the HTTP spec? Some Python library used by the project?).

Either way, it is trivial for consumers to fix that (I did just that JD557/microblog.pub#63) but, as you mentioned, it would be nice to have this behavior documented somewhere