org.codehaus.jparsec
Class Terminals

java.lang.Object
  extended by org.codehaus.jparsec.Terminals

public final class Terminals
extends Object

Provides convenient API to build lexer and parsers for terminals. The following example is a parser snippet for Java generic type expression such as List<String>:

   Terminals terms = Terminals
       .operators("?", "<", ">", ",")
       .words(Scanners.IDENTIFIER)
       .keywords("super", "extends")
       .build();
   Parser<String> typeName = Terminals.identifier();
   Parser<?> wildcardWithUpperBound = terms.phrase("?", "extends");
   ...
   parser.from(terms.tokenizer(), Scanners.WHITESPACES.optional()).parse("List<String>");
 

Author:
Ben Yu

Nested Class Summary
 class Terminals.Builder
          Builds Terminals instance by defining the words and keywords recognized.
static class Terminals.CharLiteral
          Entry point for parser and tokenizers of character literal.
static class Terminals.DecimalLiteral
          Entry point for parser and tokenizers of decimal number literal represented as String.
static class Terminals.Identifier
          Entry point for parser and tokenizers of regular identifier.
static class Terminals.IntegerLiteral
          Entry point for any arbitrary integer literal represented as a String.
static class Terminals.LongLiteral
          Entry point for parser and tokenizers of integral number literal represented as Long.
static class Terminals.ScientificNumberLiteral
          Entry point for parser and tokenizers of scientific notation literal.
static class Terminals.StringLiteral
          Entry point for parser and tokenizers of string literal.
 
Field Summary
static Parser<String> RESERVED
          Parser that recognizes reserved word tokens.
 
Method Summary
static Terminals caseInsensitive(Parser<String> wordScanner, String[] ops, String[] keywords)
          Deprecated. Use operators(ops) .words(wordScanner) .caseInsensitiveKeywords(keywords) .build() instead.
static Terminals caseInsensitive(Parser<String> wordScanner, String[] ops, String[] keywords, Map<String,?> wordMap)
          Deprecated. Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .caseInsensitiveKeywords(keywords) .build() instead.
static Terminals caseInsensitive(String[] ops, String[] keywords)
          Deprecated. Use operators(ops) .words(Scanners.IDENTIFIER) .caseInsensitiveKeywords(keywords) .build() instead.
static Terminals caseSensitive(Parser<String> wordScanner, String[] ops, String[] keywords)
          Deprecated. Use operators(ops) .words(wordScanner) .keywords(keywords) .build() instead.
static Terminals caseSensitive(Parser<String> wordScanner, String[] ops, String[] keywords, Map<String,?> wordMap)
          Deprecated. Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .keywords(keywords) .build() instead.
static Terminals caseSensitive(String[] ops, String[] keywords)
          Deprecated. Use operators(ops) .words(Scanners.IDENTIFIER) .keywords(keywords) .build() instead.
static Parser<String> fragment(Object... tags)
          Returns a Parser that recognizes Tokens.Fragment token values tagged with one of tags.
static Parser<String> identifier()
          Returns a Parser that recognizes identifiers (a.k.a words, variable names etc).
static Terminals operators(Collection<String> ops)
          Returns a Terminals object for lexing the operators with names specified in ops.
static Terminals operators(String... ops)
          Returns a Terminals object for lexing the operators with names specified in ops.
 Parser<?> phrase(String... tokenNames)
          A Parser that recognizes a sequence of tokens identified by tokenNames, as an atomic step.
 Parser<Token> token(String... tokenNames)
          A Parser that recognizes a token identified by any of tokenNames.
 Parser<Token> token(String tokenName)
          A Parser that recognizes the token identified by tokenName.
 Parser<?> tokenizer()
          Returns the tokenizer that tokenizes all terminals (operators, keywords, identifiers etc.) managed in this instance.
 Terminals.Builder words(Parser<String> wordScanner)
          Starts to build a new Terminals instance that recognizes words not already recognized by this Terminals instance (typically operators).
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

RESERVED

public static final Parser<String> RESERVED
Parser that recognizes reserved word tokens. i.e. Tokens.Fragment tokens tagged as Tokens.Tag.RESERVED. Tokens.Fragment.text() is returned as parser result.

Method Detail

caseInsensitive

@Deprecated
public static Terminals caseInsensitive(String[] ops,
                                                   String[] keywords)
Deprecated. Use operators(ops) .words(Scanners.IDENTIFIER) .caseInsensitiveKeywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

A word is defined as an alphanumeric string that starts with [_a - zA - Z], with 0 or more [0 - 9_a - zA - Z] following.

Parameters:
ops - the operator names.
keywords - the keyword names.
Returns:
the Terminals instance.

caseSensitive

@Deprecated
public static Terminals caseSensitive(String[] ops,
                                                 String[] keywords)
Deprecated. Use operators(ops) .words(Scanners.IDENTIFIER) .keywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

A word is defined as an alphanumeric string that starts with [_a - zA - Z], with 0 or more [0 - 9_a - zA - Z] following.

Parameters:
ops - the operator names.
keywords - the keyword names.
Returns:
the Terminals instance.

caseInsensitive

@Deprecated
public static Terminals caseInsensitive(Parser<String> wordScanner,
                                                   String[] ops,
                                                   String[] keywords)
Deprecated. Use operators(ops) .words(wordScanner) .caseInsensitiveKeywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

Parameters:
wordScanner - the scanner that returns a word in the language.
ops - the operator names.
keywords - the keyword names.
Returns:
the Terminals instance.

caseSensitive

@Deprecated
public static Terminals caseSensitive(Parser<String> wordScanner,
                                                 String[] ops,
                                                 String[] keywords)
Deprecated. Use operators(ops) .words(wordScanner) .keywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

Parameters:
wordScanner - the scanner that returns a word in the language.
ops - the operator names.
keywords - the keyword names.
Returns:
the Terminals instance.

caseInsensitive

@Deprecated
public static Terminals caseInsensitive(Parser<String> wordScanner,
                                                   String[] ops,
                                                   String[] keywords,
                                                   Map<String,?> wordMap)
Deprecated. Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .caseInsensitiveKeywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

Parameters:
wordScanner - the scanner that returns a word in the language.
ops - the operator names.
keywords - the keyword names.
wordMap - maps the text to a token value for non-keywords recognized by wordScanner.
Returns:
the Terminals instance.

caseSensitive

@Deprecated
public static Terminals caseSensitive(Parser<String> wordScanner,
                                                 String[] ops,
                                                 String[] keywords,
                                                 Map<String,?> wordMap)
Deprecated. Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .keywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

Parameters:
wordScanner - the scanner that returns a word in the language.
ops - the operator names.
keywords - the keyword names.
wordMap - maps the text to a token value for non-keywords recognized by wordScanner.
Returns:
the Terminals instance.

operators

public static Terminals operators(String... ops)
Returns a Terminals object for lexing the operators with names specified in ops. Operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. For example, to get the parser for operator "?", simply call token("?").

If words and keywords need to be parsed, they can be configured via words(org.codehaus.jparsec.Parser).

Parameters:
ops - the operator names.
Returns:
the Terminals instance.

operators

public static Terminals operators(Collection<String> ops)
Returns a Terminals object for lexing the operators with names specified in ops. Operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. For example, to get the parser for operator "?", simply call token("?").

If words and keywords need to be parsed, they can be configured via words(org.codehaus.jparsec.Parser).

Parameters:
ops - the operator names.
Returns:
the Terminals instance.
Since:
2.2

words

public Terminals.Builder words(Parser<String> wordScanner)
Starts to build a new Terminals instance that recognizes words not already recognized by this Terminals instance (typically operators).

By default identifiers are recognized through identifier() during token-level parsing phase. Use Terminals.Builder.tokenizeWordsWith(org.codehaus.jparsec.functors.Map) to tokenize differently, and choose an alternative token-level parser accordingly.

Parameters:
wordScanner - defines words recognized by the new instance
Since:
2.2

identifier

public static Parser<String> identifier()
Returns a Parser that recognizes identifiers (a.k.a words, variable names etc). Equivalent to Terminals.Identifier.PARSER.

Since:
2.2

fragment

public static Parser<String> fragment(Object... tags)
Returns a Parser that recognizes Tokens.Fragment token values tagged with one of tags.


tokenizer

public Parser<?> tokenizer()
Returns the tokenizer that tokenizes all terminals (operators, keywords, identifiers etc.) managed in this instance.


phrase

public Parser<?> phrase(String... tokenNames)
A Parser that recognizes a sequence of tokens identified by tokenNames, as an atomic step.


token

public Parser<Token> token(String... tokenNames)
A Parser that recognizes a token identified by any of tokenNames.


token

public Parser<Token> token(String tokenName)
A Parser that recognizes the token identified by tokenName.



Copyright © 2014. All rights reserved.