comprehensive discussion of handling white space

[brainchild0]

Background

Every space character in plain-text documents generally affects the position of the text that follows. In contrast, space characters in Markdown generally signify a word boundary, which may be displayed using a space character in some presentations, such as in an editing pane, but is not literally a character. Other white-space characters, such as line breaks, also have special significance, following rules of the language, different from their effect in plain text documents.

Because Mark Text is a Markdown editor, it should follow the rules of the language relating to handling white-space characters.

Issues

Display

In Markdown, sequences of space characters are interpreted as word boundaries. In CommonMark and many other dialects, a single line break separating non-empty lines (lines with non-white space characters) is also interpreted as a word boundary. In all dialects, any sequence of one or more empty lines (lines with only white-space characters) is interpreted as a paragraph break, and every paragraph contains at least one non-white space character.

Mark Text generally displays a line break that appears in a source document as a new line of text in the editing buffer. It generally displays contiguous space characters as additional space between words. It generally displays contiguous empty lines as additional vertical space between paragraphs.

Considering that Markdown collapses contiguous whitespace within a paragraph into a single word boundary or paragraph break, the current behavior of Mark Text may be undesirable.

Insertion

Further behaviors relating to handling of white space may not optimally represent user intention.

Generally, if a user pastes into a document a sequence of text with no paragraph breaks or block transitions, then the desired effect is that word boundaries separate the the start and end of the inserted sequence from the rest of the paragraph.

Generally, if a user pastes into a document a sequence of text with paragraph breaks or block transitions, then the desired effect is that paragraph breaks or block transitions separate the start and end of the inserted sequence from the rest of document.

Neither effect is supported automatically by Mark Text. The user is required to clean the overall text after pasting a block, in order to achieve desired appearance in the editing buffer.

Other

If a user strikes the return key a single time, modified by the shift key, then the desired effect may be to display a line break. In CommonMark and many dialects, the display of a line break occurs only if preceded by two or more space characters. In a graphical editor, the user may wish not to be required manually to insert these space characters in order to achieve the desired display for the document.

Questions

The discrepancies described above suggest the need to evaluate the optimal approach for handling white space, in particular, in each of the following scenarios:

1. Extra white space characters inserted through the keyboard.
2. Extra white space characters between pasted text and the text already in the document, at the location of insertion.
3. Lack of white space characters between pasted text and text already in the document, at the location of insertion.
4. Keyboard input of shift-return when the caret is not preceded by at least two space characters.

Additionally, it may be useful to evaluate whether to include a feature for linting white space in a document, and how such functionality might relate to the issues in #2189.

[brainchild0]

See also #1849.