Here is the modified BNF syntax for tokens.
Tokens (except for <string>) may not contain embedded
whitespace.
|
alpha-char
|
|
digit
|
|
exponent-char
|
|
guid
|
|
hex-digit
|
|
mantissa
|
|
named-char
|
|
non-negative-integer
|
|
number
|
|
octal-digit
|
|
sign
|
|
string
|
|
string-char
|
|
symbol
|
|
symbol-char
|
|
token
|
More
information
Text file format syntax.