Filler Syntax

In the following presentation we only assume an ASCII encoding. The token syntax here is thus defined based on the 7-bit characters from ASCII. There is a separate section that shows how the token syntax is extended to Unicode. The tokens can be interspersed with layout characters and comments. There are two kinds of comments, namely line comments and block comments. A line comment starts with “%” and ends with an end of line. A block comments starts with “/*” and continues until the closing “*/”:

filler         --> { layout
| linecomment
| blockcomment }.
layout         --> "\0\" ... " ".
linecomment    --> "%" { char } eol.
eol --> "\n".
blockcomment   --> "/*" { char } "*/".

Examples:

% hanoi: Solves the towers of hanoi problem      % is a line comment
/* hanoi:
Solves the towers of hanoi problem */    % is a block comment

Besides the space character all ASCII control characters also count as layout characters. The underlying streams allow the transmission of the null character (NUL) (‘\0\’). The null character does not indicate the end of a character stream or string. The encoding of a resource might use character sequences such as carriage return (CR) (‘\r’), carriage return line feed (CRLF) (‘\r\n’) or line feed (LF) (‘\n’) to indicate the end of a line. These are made visible by a single line feed (LF) (‘\n’) through the character streams.

Comments