Column of the last character of the Token.
Line of the last character of the Token.
Offset of the last character of the Token.
The textual representation of the Token as it appeared in the text.
this marks if a Token does not really exist and has been inserted "artificially" during parsing in rule error recovery.
Custom Payload value, this is an optional feature of Custom Token Patterns For additional details see the docs: https://chevrotain.io/docs/guide/custom_token_patterns.html#custom-payloads
Column of the first character of the Token.
Line of the first character of the Token.
Offset of the first character of the Token.
The actual Token Type of this Token "instance" This is the same Object returned by the "createToken" API. This property is very useful for debugging the Lexing and Parsing phases.
An number index representing the type of the Token use
Generated using TypeDoc
Things to note:
A lineTerminator as the last character does not effect the Token's line numbering. In other words a new line only starts after a line terminator.
A Token's image is it's literal text. e.g unicode escaping is untouched.