Package org.eclipse.jface.text.rules

Class RuleBasedScanner

java.lang.Object
org.eclipse.jface.text.rules.RuleBasedScanner
All Implemented Interfaces:
ICharacterScanner, ITokenScanner
Direct Known Subclasses:
BufferedRuleBasedScanner
public class RuleBasedScanner extends Object implements ICharacterScanner, ITokenScanner
A generic scanner which can be "programmed" with a sequence of rules. The scanner is used to get the next token by evaluating its rule in sequence until one is successful. If a rule returns a token which is undefined, the scanner will proceed to the next rule. Otherwise the token provided by the rule will be returned by the scanner. If no rule returned a defined token, this scanner returns a token which returns true when calling isOther, unless the end of the file is reached. In this case the token returns true when calling isEOF. Originally since unknown version, but moved to org.eclipse.text in 3.14
Since:
3.14
See Also:

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected int
    fColumn
    The cached column of the current scanner position
    protected IToken
    fDefaultReturnToken
    The token to be returned by default if no rule fires
    protected char[][]
    fDelimiters
    The cached legal line delimiters of the document
    protected IDocument
    fDocument
    The document to be scanned
    protected int
    fOffset
    The offset of the next character to be read
    protected int
    fRangeEnd
    The end offset of the range to be scanned
    protected IRule[]
    fRules
    The list of rules of this scanner
    protected int
    fTokenOffset
    The offset of the last read token
    protected static final int
    UNDEFINED
    Internal setting for the un-initialized column cache.

    Fields inherited from interface org.eclipse.jface.text.rules.ICharacterScanner

    EOF

  • Constructor Summary

    Constructors
    Constructor
    Description
    RuleBasedScanner()
    Creates a new rule based scanner which does not have any rule.

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    getColumn()
    Returns the column of the character scanner.
    char[][]
    getLegalLineDelimiters()
    Provides rules access to the legal line delimiters.
    int
    getTokenLength()
    Returns the length of the last token read by this scanner.
    int
    getTokenOffset()
    Returns the offset of the last token read by this scanner.
    IToken
    nextToken()
    Returns the next token in the document.
    int
    read()
    Returns the next character or EOF if end of file has been reached
    void
    setDefaultReturnToken(IToken defaultReturnToken)
    Configures the scanner's default return token.
    void
    setRange(IDocument document, int offset, int length)
    Configures the scanner by providing access to the document range that should be scanned.
    void
    setRules(IRule... rules)
    Configures the scanner with the given sequence of rules.
    void
    unread()
    Rewinds the scanner before the last read character.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • fRules

      protected IRule[] fRules
      The list of rules of this scanner

    • fDefaultReturnToken

      protected IToken fDefaultReturnToken
      The token to be returned by default if no rule fires

    • fDocument

      protected IDocument fDocument
      The document to be scanned

    • fDelimiters

      protected char[][] fDelimiters
      The cached legal line delimiters of the document

    • fOffset

      protected int fOffset
      The offset of the next character to be read

    • fRangeEnd

      protected int fRangeEnd
      The end offset of the range to be scanned

    • fTokenOffset

      protected int fTokenOffset
      The offset of the last read token

    • fColumn

      protected int fColumn
      The cached column of the current scanner position

    • UNDEFINED

      protected static final int UNDEFINED
      Internal setting for the un-initialized column cache.
      See Also:

  • Constructor Details

    • RuleBasedScanner

      public RuleBasedScanner()
      Creates a new rule based scanner which does not have any rule.

  • Method Details

    • setRules

      public void setRules(IRule... rules)
      Configures the scanner with the given sequence of rules.
      Parameters:
      rules - the sequence of rules controlling this scanner

    • setDefaultReturnToken

      public void setDefaultReturnToken(IToken defaultReturnToken)
      Configures the scanner's default return token. This is the token which is returned when none of the rules fired and EOF has not been reached.
      Parameters:
      defaultReturnToken - the default return token
      Since:
      2.0

    • setRange

      public void setRange(IDocument document, int offset, int length)
      Description copied from interface: ITokenScanner
      Configures the scanner by providing access to the document range that should be scanned.
      Specified by:
      setRange in interface ITokenScanner
      Parameters:
      document - the document to scan
      offset - the offset of the document range to scan
      length - the length of the document range to scan

    • getTokenOffset

      public int getTokenOffset()
      Description copied from interface: ITokenScanner
      Returns the offset of the last token read by this scanner.
      Specified by:
      getTokenOffset in interface ITokenScanner
      Returns:
      the offset of the last token read by this scanner

    • getTokenLength

      public int getTokenLength()
      Description copied from interface: ITokenScanner
      Returns the length of the last token read by this scanner.
      Specified by:
      getTokenLength in interface ITokenScanner
      Returns:
      the length of the last token read by this scanner

    • getColumn

      public int getColumn()
      Description copied from interface: ICharacterScanner
      Returns the column of the character scanner.
      Specified by:
      getColumn in interface ICharacterScanner
      Returns:
      the column of the character scanner

    • getLegalLineDelimiters

      public char[][] getLegalLineDelimiters()
      Description copied from interface: ICharacterScanner
      Provides rules access to the legal line delimiters. The returned object may not be modified by clients.
      Specified by:
      getLegalLineDelimiters in interface ICharacterScanner
      Returns:
      the legal line delimiters

    • nextToken

      public IToken nextToken()
      Description copied from interface: ITokenScanner
      Returns the next token in the document.
      Specified by:
      nextToken in interface ITokenScanner
      Returns:
      the next token in the document

    • read

      public int read()
      Description copied from interface: ICharacterScanner
      Returns the next character or EOF if end of file has been reached
      Specified by:
      read in interface ICharacterScanner
      Returns:
      the next character or EOF

    • unread

      public void unread()
      Description copied from interface: ICharacterScanner
      Rewinds the scanner before the last read character.
      Specified by:
      unread in interface ICharacterScanner