Class RuleBasedScanner

  • All Implemented Interfaces:
    ICharacterScanner, ITokenScanner
    Direct Known Subclasses:
    BufferedRuleBasedScanner

    public class RuleBasedScanner
    extends Object
    implements ICharacterScanner, ITokenScanner
    A generic scanner which can be "programmed" with a sequence of rules. The scanner is used to get the next token by evaluating its rule in sequence until one is successful. If a rule returns a token which is undefined, the scanner will proceed to the next rule. Otherwise the token provided by the rule will be returned by the scanner. If no rule returned a defined token, this scanner returns a token which returns true when calling isOther, unless the end of the file is reached. In this case the token returns true when calling isEOF.
    See Also:
    IRule
    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected int fColumn
      The cached column of the current scanner position
      protected IToken fDefaultReturnToken
      The token to be returned by default if no rule fires
      protected char[][] fDelimiters
      The cached legal line delimiters of the document
      protected IDocument fDocument
      The document to be scanned
      protected int fOffset
      The offset of the next character to be read
      protected int fRangeEnd
      The end offset of the range to be scanned
      protected IRule[] fRules
      The list of rules of this scanner
      protected int fTokenOffset
      The offset of the last read token
      protected static int UNDEFINED
      Internal setting for the un-initialized column cache.
    • Constructor Summary

      Constructors 
      Constructor Description
      RuleBasedScanner()
      Creates a new rule based scanner which does not have any rule.
    • Field Detail

      • fRules

        protected IRule[] fRules
        The list of rules of this scanner
      • fDefaultReturnToken

        protected IToken fDefaultReturnToken
        The token to be returned by default if no rule fires
      • fDocument

        protected IDocument fDocument
        The document to be scanned
      • fDelimiters

        protected char[][] fDelimiters
        The cached legal line delimiters of the document
      • fOffset

        protected int fOffset
        The offset of the next character to be read
      • fRangeEnd

        protected int fRangeEnd
        The end offset of the range to be scanned
      • fTokenOffset

        protected int fTokenOffset
        The offset of the last read token
      • fColumn

        protected int fColumn
        The cached column of the current scanner position
      • UNDEFINED

        protected static final int UNDEFINED
        Internal setting for the un-initialized column cache.
        See Also:
        Constant Field Values
    • Constructor Detail

      • RuleBasedScanner

        public RuleBasedScanner()
        Creates a new rule based scanner which does not have any rule.
    • Method Detail

      • setRules

        public void setRules​(IRule[] rules)
        Configures the scanner with the given sequence of rules.
        Parameters:
        rules - the sequence of rules controlling this scanner
      • setDefaultReturnToken

        public void setDefaultReturnToken​(IToken defaultReturnToken)
        Configures the scanner's default return token. This is the token which is returned when none of the rules fired and EOF has not been reached.
        Parameters:
        defaultReturnToken - the default return token
        Since:
        2.0
      • setRange

        public void setRange​(IDocument document,
                             int offset,
                             int length)
        Description copied from interface: ITokenScanner
        Configures the scanner by providing access to the document range that should be scanned.
        Specified by:
        setRange in interface ITokenScanner
        Parameters:
        document - the document to scan
        offset - the offset of the document range to scan
        length - the length of the document range to scan
      • getTokenOffset

        public int getTokenOffset()
        Description copied from interface: ITokenScanner
        Returns the offset of the last token read by this scanner.
        Specified by:
        getTokenOffset in interface ITokenScanner
        Returns:
        the offset of the last token read by this scanner
      • getTokenLength

        public int getTokenLength()
        Description copied from interface: ITokenScanner
        Returns the length of the last token read by this scanner.
        Specified by:
        getTokenLength in interface ITokenScanner
        Returns:
        the length of the last token read by this scanner
      • getColumn

        public int getColumn()
        Description copied from interface: ICharacterScanner
        Returns the column of the character scanner.
        Specified by:
        getColumn in interface ICharacterScanner
        Returns:
        the column of the character scanner
      • getLegalLineDelimiters

        public char[][] getLegalLineDelimiters()
        Description copied from interface: ICharacterScanner
        Provides rules access to the legal line delimiters. The returned object may not be modified by clients.
        Specified by:
        getLegalLineDelimiters in interface ICharacterScanner
        Returns:
        the legal line delimiters
      • nextToken

        public IToken nextToken()
        Description copied from interface: ITokenScanner
        Returns the next token in the document.
        Specified by:
        nextToken in interface ITokenScanner
        Returns:
        the next token in the document
      • read

        public int read()
        Description copied from interface: ICharacterScanner
        Returns the next character or EOF if end of file has been reached
        Specified by:
        read in interface ICharacterScanner
        Returns:
        the next character or EOF