olski.markdown

Markdown in, Polish prose out.

The grammar needs a guarantee plain text gives and Markdown does not: every character is prose. This produces it, by dropping the apparatus and by joining what the renderer would have joined.

Moduł należy do olski, bo dokument czyta olski-check, a nie krok przed nim; czemu granica pakietów pada tutaj, mówi harness/__init__.py.

The decisions below run through the whole module.

A parser says where a construct is; this module says what to do with it. Which characters are markup is a question about CommonMark, and markdown-it-py answers it, so that what no pattern settles — which of two adjacent emphases a marker closes, whether a run of backticks opens a fence or a code span — is settled by something tested against the specification. What stays here is the half a renderer has no opinion on: which constructs are apparatus, and what a construct leaves behind when it goes.

Inline markup is replaced by the text it wrapped, never deleted. A deletion leaves the space that stood in front of it, and the sentence then reaches the grammar with a gap nobody typed: docs/extraction.md holds what that cost the two extractions written before this one. Where a construct has no text to leave behind, the space in front of it goes with it.

A line is a line of source, not a line of the page. Everything a paragraph holds is joined with single spaces, because that is what a renderer does with a newline inside one, and where the author's editor wrapped then leaves no trace in what comes out.

MARKDOWN_SUFFIX = '.md' module-attribute

PARSER = MarkdownIt('commonmark').enable(['table', 'strikethrough']) module-attribute

FRONTMATTER = re.compile('\\A---[ \\t]*\\n.*?\\n(?:---|\\.\\.\\.)[ \\t]*(?:\\n|\\Z)', re.DOTALL) module-attribute

CONTAINERS = frozenset({'blockquote', 'bullet_list', 'ordered_list', 'list_item'}) module-attribute

LISTS = frozenset({'bullet_list', 'ordered_list'}) module-attribute

LITERAL = frozenset({'text', 'text_special', 'code_inline'}) module-attribute

WRAPPING = frozenset({'link', 'image', 'em', 'strong', 's'}) module-attribute

BREAKS = frozenset({'softbreak', 'hardbreak'}) module-attribute

prose(text)

Return the prose of a Markdown document, one paragraph per line.

Blank lines separate paragraphs, so that a sentence does not run from one paragraph into the next, and nothing else in the result is a line break.

Source code in olski/markdown.py

def prose(text: str) -> str:
    """Return the prose of a Markdown document, one paragraph per line.

    Blank lines separate paragraphs, so that a sentence does not run from one
    paragraph into the next, and nothing else in the result is a line break.
    """
    root = SyntaxTreeNode(PARSER.parse(FRONTMATTER.sub("", text)))
    paragraphs = (paragraph.strip() for paragraph in _paragraphs(_without_trailing_links(root)))
    #  A block can come out empty — an image with no description is a paragraph
    #  of nothing — and an empty line between two others would read as a break.
    body = "\n\n".join(paragraph for paragraph in paragraphs if paragraph)
    return body + "\n" if body else ""