ITokenScanner (Eclipse Platform API Specification)

All Known Subinterfaces:

IPartitionTokenScanner

All Known Implementing Classes:

BufferedRuleBasedScanner, RuleBasedPartitionScanner, RuleBasedScanner
```
public interface ITokenScanner
```
A token scanner scans a range of a document and reports about the token it finds. A scanner has state. When asked, the scanner returns the offset and the length of the last found token.

Since:

2.0

See Also:

IToken

Method Summary

All Methods Instance Methods Abstract Methods
Modifier and Type	Method	Description
`int`	`getTokenLength()`	Returns the length of the last token read by this scanner.
`int`	`getTokenOffset()`	Returns the offset of the last token read by this scanner.
`IToken`	`nextToken()`	Returns the next token in the document.
`void`	`setRange(IDocument document, int offset, int length)`	Configures the scanner by providing access to the document range that should be scanned.

- Method Detail
  - setRange
```
void setRange(IDocument document,
              int offset,
              int length)
```
    Configures the scanner by providing access to the document range that should be scanned.
    
    Parameters:
    
    document - the document to scan
    
    offset - the offset of the document range to scan
    
    length - the length of the document range to scan
  - nextToken
```
IToken nextToken()
```
    Returns the next token in the document.
    
    Returns:
    
    the next token in the document
  - getTokenOffset
```
int getTokenOffset()
```
    Returns the offset of the last token read by this scanner.
    
    Returns:
    
    the offset of the last token read by this scanner
  - getTokenLength
```
int getTokenLength()
```
    Returns the length of the last token read by this scanner.
    
    Returns:
    
    the length of the last token read by this scanner