org.basex.util.ft
Class Tokenizer
java.lang.Object
org.basex.util.ft.FTIterator
org.basex.util.ft.Tokenizer
- All Implemented Interfaces:
- java.lang.Comparable<org.basex.util.ft.LanguageImpl>, java.util.Iterator<FTSpan>
- Direct Known Subclasses:
- JapaneseTokenizer, WesternTokenizer
public abstract class Tokenizer
- extends FTIterator
Abstract tokenizer.
- Author:
- BaseX Team 2005-12, BSD License, Jens Erat
Field Summary |
static java.util.LinkedList<Tokenizer> |
IMPL
List of available tokenizers. |
Method Summary |
int |
compareTo(org.basex.util.ft.LanguageImpl o)
|
boolean |
equals(java.lang.Object o)
|
int |
hashCode()
|
protected abstract byte |
prec()
Returns the precedence of the processor. |
static boolean |
supportFor(Language l)
Checks if the language is supported by the available tokenizers. |
boolean |
supports(Language ln)
Checks if the specified language is supported. |
Methods inherited from class java.lang.Object |
clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface java.util.Iterator |
hasNext, next |
IMPL
public static final java.util.LinkedList<Tokenizer> IMPL
- List of available tokenizers.
Tokenizer
public Tokenizer()
supportFor
public static boolean supportFor(Language l)
- Checks if the language is supported by the available tokenizers.
- Parameters:
l
- language to be found
- Returns:
- result of check
prec
protected abstract byte prec()
- Returns the precedence of the processor. If two language implementations
support the same language, the processor with the higher precedence
will be selected.
- Returns:
- precedence
supports
public boolean supports(Language ln)
- Checks if the specified language is supported.
- Parameters:
ln
- language
- Returns:
- true if language is supported
equals
public final boolean equals(java.lang.Object o)
- Overrides:
equals
in class java.lang.Object
compareTo
public final int compareTo(org.basex.util.ft.LanguageImpl o)
- Specified by:
compareTo
in interface java.lang.Comparable<org.basex.util.ft.LanguageImpl>
hashCode
public int hashCode()
- Overrides:
hashCode
in class java.lang.Object