Interface IToken

Things to note:

  • The offset range is inclusive to exclusive.

  • A lineTerminator as the last character does not effect the Token's line numbering. In other words a new line only starts after a line terminator.

  • A Token's image is it's literal text. e.g unicode escaping is untouched.

Hierarchy

  • IToken

Properties

endColumn?: number

Column of the last character of the Token. 1-indexed. See also startColumn.

endLine?: number

Line of the last character of the Token. 1-indexed. Will be the same as startLine for single-line tokens.

endOffset?: number

Offset of the last character of the Token. 0-indexed. Note that this points at the last character, not the end of the token, so the original image would be input.substring(token.startOffset, token.endOffset + 1).

image: string

The textual representation of the Token as it appeared in the text.

isInsertedInRecovery?: boolean

this marks if a Token does not really exist and has been inserted "artificially" during parsing in rule error recovery.

payload?: any

Custom Payload value, this is an optional feature of Custom Token Patterns For additional details see the docs: https://chevrotain.io/docs/guide/custom_token_patterns.html#custom-payloads

startColumn?: number

Column of the first character of the Token. 1-indexed.

For token foo in the following line, startColumn will be 3 and endColumn will be 5.

a foo
123456
startLine?: number

Line of the first character of the Token. 1-indexed.

startOffset: number

Offset of the first character of the Token. 0-indexed.

tokenType: TokenType

The actual Token Type of this Token "instance" This is the same Object returned by the "createToken" API. This property is very useful for debugging the Lexing and Parsing phases.

tokenTypeIdx: number

An number index representing the type of the Token use to get the Token Type from a token "instance"

Generated using TypeDoc