TokenType
public protocol TokenType
TokenType
s are generated when rules are matched (usually, sometime a rule just advances the scan-head). Tokens with a rawValue
of -1 are considered transient, meaning that they should not be included in any construction of an AST. However, they may provide context to the AST.