|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.basex.util.ft.FTIterator
org.basex.util.ft.FTLexer
public final class FTLexer
Performs full-text lexing on token. Calls tokenizers, stemmers matching to full-text options to achieve this.
Constructor Summary | |
---|---|
FTLexer()
Constructor, using the default full-text options. |
|
FTLexer(FTOpt opt)
Default constructor. |
Method Summary | |
---|---|
int |
count()
Returns total number of tokens. |
FTOpt |
ftOpt()
Returns the full-text options. |
byte[] |
get()
Returns the original token. |
boolean |
hasNext()
|
int[][] |
info()
Gets full-text info for the specified token; needed for visualizations. |
void |
init()
Initializes the iterator. |
FTLexer |
init(byte[] txt)
Initializes the iterator. |
static StringList |
languages()
Lists all languages for which tokenizers and stemmers are available. |
FTSpan |
next()
|
byte[] |
nextToken()
Returns the next token. |
boolean |
paragraph()
Is paragraph? Does not have to be implemented by all tokenizers. |
int |
pos(int w,
FTUnit u)
Calculates a position value, dependent on the specified unit. |
FTLexer |
sc()
Sets the special character flag. |
byte[] |
text()
Returns the text to be processed. |
IndexType |
type()
Returns the index type. |
Methods inherited from class org.basex.util.ft.FTIterator |
---|
remove |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public FTLexer()
XMLSerializer
, FTFilter
, and the map visualizations.
public FTLexer(FTOpt opt)
opt
- full-text optionsMethod Detail |
---|
public FTLexer sc()
public void init()
public FTLexer init(byte[] txt)
FTIterator
init
in class FTIterator
txt
- text
public boolean hasNext()
hasNext
in interface java.util.Iterator<FTSpan>
public FTSpan next()
next
in interface java.util.Iterator<FTSpan>
public byte[] nextToken()
FTIterator
Iterator.next()
to avoid the creation of new FTSpan
instances.
nextToken
in class FTIterator
public int count()
public IndexType type()
IndexToken
type
in interface IndexToken
public byte[] get()
IndexToken
;
use next()
or nextToken()
if not using this interface.
get
in interface IndexToken
public FTOpt ftOpt()
null
.
public byte[] text()
public boolean paragraph()
public int pos(int w, FTUnit u)
w
- word positionu
- unit
public int[][] info()
Tokenizer.info()
for more info.
public static StringList languages()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |