Compiler/lexical analyzer: Difference between revisions

* Zero or more whitespace characters or comments are allowed between any two tokens, with the exceptions noted below.

* "longest token matching" is used to resolve conflicts (e.g., in order to match '''<=''' as a single token rather than the two tokens '''<''' and '''=''').

* Whitespace is only required in the following situations:

** To distinguish between keywords:

*** ifprint - is recognized as an identifier, instead of the keywords '''if''' and '''print'''.

** To distinguish between keywords and integers:

*** 42fred - is an invalid number or invalid identifier.

* Whitespace is not allowed between:

** Multi-character operators: These cannot be recognized unless they occur without embedded whitespace: '''&&''' '''<='''.