Semantics - Embedded

TLDR

Run and Debug the source code.

Introduction

In the previous tutorial step we have implemented a parser for a "mini" SQL Select grammar. The current problem is that our parser only validates the input conforms to the grammar. In most real world use cases the parser will also have to output some result/data structure/value.

This can be accomplished using two features of the Parsing DSL:

• CONSUME will return The IToken object consumed.
• SUBRULE will return the result of the grammar rule invoked.

Enabling embedded actions

For embedded actions to work as expected we need to extend the EmbeddedActionsParser class instead of the CstParser class.

const { EmbeddedActionsParser } = require("chevrotain");

class SelectParserEmbedded extends EmbeddedActionsParser {
  constructor() {
    super(tokenVocabulary);
  }
}

Failing to disabled the CST creation would cause the Parser to return a CST of the grammar rule we invoked instead of of the expected output structure we will be creating (an AST).

Simple Example

Lets inspect a simple contrived example:

$.RULE("topRule", () => {
  let result = 0;

  $.MANY(() => {
    $.OR([
      {
        ALT: () => {
          result += $.SUBRULE($.decimalRule);
        },
      },
      {
        ALT: () => {
          result += $.SUBRULE($.IntegerRule);
        },
      },
    ]);
  });

  return result;
});

$.RULE("decimalRule", () => {
  const decimalToken = $.CONSUME(Decimal);
  return parseFloat(decimalToken.image);
});

$.RULE("IntegerRule", () => {
  const intToken = $.CONSUME(Integer);
  return parseInt(intToken.image);
});

The decimalRule and IntegerRule both return a javascript number (using parseInt/parseFloat). and the topRule adds it to the final result.

SQL Grammar

Lets go back to the mini SQL Select grammar.

For this grammar we will build a more complex data structure (an AST) instead of simply returning a number. Our selectStatement rule will now return an object with four properties:

$.RULE("selectStatement", () => {
  let select, from, where;

  select = $.SUBRULE($.selectClause);
  from = $.SUBRULE($.fromClause);
  $.OPTION(() => {
    where = $.SUBRULE($.whereClause);
  });

  return {
    type: "SELECT_STMT",
    selectClause: select,
    fromClause: from,
    // may be undefined if the OPTION was not entered.
    whereClause: where,
  };
});

Three of those properties (selectClause / fromClause / whereClause) are the results of invoking other parser rules.

Lets look at the "selectClause" rule implementation:

$.RULE("selectClause", () => {
  let columns = [];

  $.CONSUME(Select);
  $.AT_LEAST_ONE_SEP({
    SEP: Comma,
    DEF: () => {
      // accessing a token's original text via the `image` property
      columns.push($.CONSUME(Identifier).image);
    },
  });

  return {
    type: "SELECT_CLAUSE",
    columns: columns,
  };
});

In the selectClause rule we access the image property of the Identifier token returned from CONSUME and push each of these strings to the columns array.