org.basex.util.ft
Class Tokenizer

java.lang.Object
  extended by org.basex.util.ft.FTIterator
      extended by org.basex.util.ft.Tokenizer
All Implemented Interfaces:
java.lang.Comparable<org.basex.util.ft.LanguageImpl>, java.util.Iterator<FTSpan>
Direct Known Subclasses:
JapaneseTokenizer, WesternTokenizer

public abstract class Tokenizer
extends FTIterator

Abstract tokenizer.

Author:
BaseX Team 2005-12, BSD License, Jens Erat

Field Summary
static java.util.LinkedList<Tokenizer> IMPL
          List of available tokenizers.
 
Constructor Summary
Tokenizer()
           
 
Method Summary
 int compareTo(org.basex.util.ft.LanguageImpl o)
           
 boolean equals(java.lang.Object o)
           
 int hashCode()
           
protected abstract  byte prec()
          Returns the precedence of the processor.
static boolean supportFor(Language l)
          Checks if the language is supported by the available tokenizers.
 boolean supports(Language ln)
          Checks if the specified language is supported.
 
Methods inherited from class org.basex.util.ft.FTIterator
init, nextToken, remove
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface java.util.Iterator
hasNext, next
 

Field Detail

IMPL

public static final java.util.LinkedList<Tokenizer> IMPL
List of available tokenizers.

Constructor Detail

Tokenizer

public Tokenizer()
Method Detail

supportFor

public static boolean supportFor(Language l)
Checks if the language is supported by the available tokenizers.

Parameters:
l - language to be found
Returns:
result of check

prec

protected abstract byte prec()
Returns the precedence of the processor. If two language implementations support the same language, the processor with the higher precedence will be selected.

Returns:
precedence

supports

public boolean supports(Language ln)
Checks if the specified language is supported.

Parameters:
ln - language
Returns:
true if language is supported

equals

public final boolean equals(java.lang.Object o)
Overrides:
equals in class java.lang.Object

compareTo

public final int compareTo(org.basex.util.ft.LanguageImpl o)
Specified by:
compareTo in interface java.lang.Comparable<org.basex.util.ft.LanguageImpl>

hashCode

public int hashCode()
Overrides:
hashCode in class java.lang.Object