org.basex.util.ft
Class WesternTokenizer

java.lang.Object
  extended by org.basex.util.ft.FTIterator
      extended by org.basex.util.ft.Tokenizer
          extended by org.basex.util.ft.WesternTokenizer
All Implemented Interfaces:
java.lang.Comparable<org.basex.util.ft.LanguageImpl>, java.util.Iterator<FTSpan>

public final class WesternTokenizer
extends Tokenizer

Full-text tokenizer.

Author:
BaseX Team 2005-12, BSD License, Christian Gruen

Field Summary
 
Fields inherited from class org.basex.util.ft.Tokenizer
IMPL
 
Constructor Summary
WesternTokenizer(FTOpt f)
          Constructor.
 
Method Summary
 int compareTo(org.basex.util.ft.LanguageImpl o)
           
 boolean equals(java.lang.Object o)
           
 int hashCode()
           
 boolean hasNext()
           
 WesternTokenizer init(byte[] txt)
          Initializes the iterator.
 FTSpan next()
           
 byte[] nextToken()
          Returns the next token.
protected  byte prec()
          Returns the precedence of the processor.
 boolean supports(Language ln)
          Checks if the specified language is supported.
 java.lang.String toString()
           
 
Methods inherited from class org.basex.util.ft.Tokenizer
supportFor
 
Methods inherited from class org.basex.util.ft.FTIterator
remove
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

WesternTokenizer

public WesternTokenizer(FTOpt f)
Constructor.

Parameters:
f - full-text options
Method Detail

init

public WesternTokenizer init(byte[] txt)
Description copied from class: FTIterator
Initializes the iterator.

Specified by:
init in class FTIterator
Parameters:
txt - text
Returns:
self reference

hasNext

public boolean hasNext()

next

public FTSpan next()

nextToken

public byte[] nextToken()
Description copied from class: FTIterator
Returns the next token. May be called as an alternative to Iterator.next() to avoid the creation of new FTSpan instances.

Specified by:
nextToken in class FTIterator
Returns:
token

prec

protected byte prec()
Returns the precedence of the processor. If two language implementations support the same language, the processor with the higher precedence will be selected.

Returns:
precedence

toString

public java.lang.String toString()
Overrides:
toString in class java.lang.Object

supports

public boolean supports(Language ln)
Checks if the specified language is supported.

Parameters:
ln - language
Returns:
true if language is supported

equals

public final boolean equals(java.lang.Object o)
Overrides:
equals in class java.lang.Object

compareTo

public final int compareTo(org.basex.util.ft.LanguageImpl o)
Specified by:
compareTo in interface java.lang.Comparable<org.basex.util.ft.LanguageImpl>

hashCode

public int hashCode()
Overrides:
hashCode in class java.lang.Object