public class SmalltalkTokenizer extends LanguageTokenizer
SmalltalkTokenizer class parses a text into "tokens".
The text should represent a Smalltalk program.| 修飾子とタイプ | フィールドと説明 |
|---|---|
protected int |
keywordBegin |
protected int |
keywordEnd |
static Set |
Keywords
The keywords of the Smalltalk languages.
|
COMMENT, CONSTANT, EOT, inMultiLineComment, inMultiLineConstant, KEYWORD, MULTILINE_COMMENT, MULTILINE_CONSTANT, OTHER, tokenBegin, tokenEnd| コンストラクタと説明 |
|---|
SmalltalkTokenizer(Text text,
int begin,
int end)
Construct a
SmalltalkTokenizer with the specified
text and range. |
| 修飾子とタイプ | メソッドと説明 |
|---|---|
int |
nextToken()
Parses the next token from the text of this tokenizer.
|
getPosition, getPreferredParseBegin, getPreferredParseEnd, isDigit, isLetter, isLetterOrDigit, isLowerLetter, peek, pushBack, read, setPositionclone, current, equals, first, getBeginIndex, getEndIndex, getIndex, getText, hashCode, last, next, previous, setIndexprotected int keywordBegin
protected int keywordEnd
public static final Set Keywords
public SmalltalkTokenizer(Text text, int begin, int end)
SmalltalkTokenizer with the specified
text and range.text - the text to be parsed.begin - the beginning index to parse, inclusive.end - the ending index to parse, exclusive.public int nextToken()
nextToken クラス内 LanguageTokenizerLanguageTokenizer.EOT,
LanguageTokenizer.OTHER,
LanguageTokenizer.KEYWORD,
LanguageTokenizer.CONSTANT,
LanguageTokenizer.MULTILINE_CONSTANT,
LanguageTokenizer.COMMENT,
LanguageTokenizer.MULTILINE_COMMENT