Reranking

The reranker is a collection of algorithms that detect exact and near-exact token matches in the query, lifting them to the top of the result list. It complements the core pattern recognition which always runs first. It is on by default (Query.Ranking.Rerank), in either search mode. Before 6.0 it was called Coverage, and the API's names said so (EnableCoverageReport, CoverageReport, Field.CoverageWeight); they are EnableMatchReport, MatchReport and Field.RerankWeight now.

How It Works

The search first counts the documents that hold every query word, exactly or within one or two edits (the candidates, see Query.Candidates). The reranker then scores each of them 0–65535 by how much of the query the document covers, and blends that with the pattern recognition score (Query.Ranking.Share).

How the Reranker Ranks

The score is the share of the query text the document covers, so 65535 means all of it. Among documents that cover the query equally, three small deductions decide the order:

DeductionAmountWhen
Word out of order
257 per word
An exactly matched word breaks the query's word order. Order is relative: the phrase in order anywhere in the text is in order
Not at the start
16
The document does not begin with the query's first word
Ends inside a word
8
The whole query was found as one string, but its last character is followed by more letters

When the whole query appears in the document as one string, it sets a floor under the score, by where that string starts:

Whole query foundScore at least
At the start of the document
65535
At the start of a word further in
65519
Inside a word
(q − 1) / q × 65535, where q is the query length

This is what makes search-as-you-type rank well. For knut h, the final h is too short to count as a word, but knut hamsun contains the whole query and scores 65527 (starts the document, ends inside a word), while knut faldbakken covers only knut and scores 43690.

Diacritic folding follows the TextNormalizer. In one-way mode (the default) a query is folded only against a document whose text folds, so blåbær fully matches blåbær but not blabar, while blabar fully matches both. In two-way mode both sides always fold, and the two spellings are the same query.

The settings

What the reranker reads is spread over the query's three parts (6.0; until then one CoverageSetup):

csharp
var query = new Query("knut hamsun", 20)
{
    Match = { Tolerance = Tolerance.OneEdit, JoinedWords = true }, // what counts as a hit
    Candidates = { Fallback = true },                              // who comes in
    Ranking = { Share = 0.2f, WholeQueryFloor = true },            // the order
};
SettingDefaultDescription
Match.Tolerance
TwoEdits
The most edits a word may differ by in this search (Exact, OneEdit, TwoEdits, ThreeEdits). It caps each field's own Field.Tolerance (default OneEdit), never raises it
Match.OneEditFrom, Match.TwoEditsFrom, Match.ThreeEditsFrom
3, 6, 9
The shortest word given one edit, two and three. 0 is never. Edits reach words of 64 characters at most; a longer word is met as typed only
Match.JoinedWords
true
Words written together or apart count as the word (blueberry for blue berry)
Ranking.Rerank
true
Whether the reranker orders the candidates, in either search mode (Query.Mode). false: the pattern score alone
Ranking.Share
0.2
Share × reranker score + (1 − Share) × pattern score. 1 is the reranker alone
Ranking.TypingShift
0.5
How far the share moves towards the reranker alone while the last word is being typed
Ranking.WholeQueryFloor
true
The whole query found as one string sets a floor under the score (see above)
Ranking.LongFieldFrom, Ranking.LongFieldWeightFloor
200, 0
From this length, in characters, a record's field counts less in the reranker: its weight is multiplied by this length over the field's own to the power 0.18 (735 characters count 0.79, 2 000 count 0.66), never below the floor. 0 is never
Ranking.FieldWeights
empty
Per field, its weight in the reranker in this search in place of Field.RerankWeight; 0 leaves the field out of the reranker and the word lookup
Candidates.Fallback
true
When no record holds every query word: the records holding the most of the words, then pattern matches (Result.IsFallback). false gives an empty list

Whole words, and a word as the start or end of a longer one, are always detected; the shortest word is TokenizerSetup.MinimumWordSize. CoverWholeWords, CoverPrefixSuffix, PrefixSuffixWhileTyping, MinWordSize and the per-field entries of CoverageSetup.Fields are gone, as are the cut and its settings: the list is the counted set, cut only to the rows asked for.

Common Patterns

Reranking should be kept on (Ranking.Rerank, the default) for nearly all use cases, including search-as-you-type. It improves result quality across multiple searchable fields, at some cost per search: turn it off where speed matters more than the order inside the counted set.

Agent/tool usage (only records holding every word):

csharp
query.Candidates.Fallback = false;

More candidates than the rows times four:

csharp
query.Candidates.MaxCandidates = 5000;

Constructors

csharp
MatchSetup()
csharp
CandidateSetup()
csharp
RankingSetup()

Properties

NameTypeAccessDescription
Tolerance
Tolerance
get/set
The most a word may differ by in this search:,or, the default, which leaves each field its own (): a field's own is capped by it, never raised.
Rungs
Rungs
get/set
Which rungs are counted:, the default, the first that gives hits;, every one up to the tolerance, the nearer ranked first.
OneEditFrom
int
get/set
The shortest query word given one edit, 3 by default; 0 is never. At three characters one edit already reaches a large share of the vocabulary, below it nearly everything.
TwoEditsFrom
int
get/set
The shortest query word given two edits, 6 by default; 0 is never. Two edits are a small share of a long word and a large one of a short word (foris two in eleven).
ThreeEditsFrom
int
get/set
The shortest query word given three edits, 9 by default; 0 is never. Only where the search'sand the field's are. A word of 10 letters or more typically meets one term within three edits; one of 6 or 7 letters meets dozens, and at worst thousands. Edits of any number reach words of 64 characters at most; a longer word is met exactly only ().
CursorAt
int?
get/set
TextWhere the cursor is in, as a character index: the word it is in or right after is the word being typed, met by its start as well as within its edits; every other word is finished, met within its edits only. A cursor at 0 or right after a delimiter is in no word. Null, the default: the last word is being typed when no delimiter follows it. A search box that knows the cursor sends it, so that a user going back to mend an earlier word ("kyll filet", the cursor after "kyll") gets that word's starts.
TypingPrefixFrom
int
get/set
The word being typed is also met as the start of a longer word from this many characters, 2 by default ("le" finds Leonardo); 0 is never.
TypingPrefixOneEditFrom
int
get/set
That start within one edit from this many characters, 6 by default; 0 is never.
JoinedWords
bool
get/set
Words written together or apart count as the word ("blueberry" for "blue berry", "hotsauce" for "hot sauce"), on by default.
CandidatesPerRow
int
get/set
The candidates a search takes per row asked for, 4 by default: the bound is the rows times this, at most 5 000, never fewer than the rows. Below 1 is taken as 1.
MaxCandidates
int?
get/set
CandidatesPerRowThe bound itself, when set: how many candidates the search takes, outright, in place of the rows timesand past the floor of 100 and the roof of 5 000 the engine keeps that between. Null, the default. Set it whenis aboveand more of the counted should be ranked.
MinCandidates
int
get/set
The fewest candidates a search takes when the rows timesare fewer, 100 by default: a search asking for few rows still has candidates enough to rank. Taken between 1 and the roof of 5 000;, when set, sets the bound outright past it.
NoFallbackForNumbersFrom
int
get/set
No fallback for a number no record holds: a query that is one number of at least this many digits (a final X allowed, for an ISBN-10) and that no record holds answers asfalse does, no records and a count of 0, rather than the records sharing some of its digits. 0, the default, is off. Only the word lookup can say that no record holds a word, so the field holding the numbers must be matched by its words (); a number held only in a field matched by pattern is never found by the lookup, and with this on its searches would answer empty.
IndexTerms
IndexTerms
get/set
BothWhich of the index's terms BM25F scores with:, the default, as the index holds them; the words alone; or the pieces alone. The index is not rebuilt: a term knows whether it is a word.
Fallback
bool
get/set
IsFallbackWhen no document holds every query word: the fallback, the documents holding the most of the words, then BM25F's best (). On by default; off gives an empty list, the counts still said.
FillWithMostWords
bool
get/set
When too few documents hold every query word to fill the rows asked for: below the counted documents and the next rungs' fill, the documents holding the most of the words, those lacking one word first and, within that, those lacking the commonest, shown and not counted (;, the rungs and the facets stay the counted set's). Off by default: it gains most in a product search with few hits, and is noise below the one record a title search wants ("matrix reloaded": many films hold one of the words). Apart from, so a dataset can keep the fallback for a query with a large typo and leave short pages short.
FillReach
int
get/set
How many rungs past the counted one the word lookup climbs to fill a page the counted documents leave short (with), shown below them and not counted (): 0, the default, no fill, the page as short as the counted documents leave it; 1 the next rung only ("chili" counted, "chilli" below it; "dark night" counted, The Dark Knight below it); 2 up to two edits. Below 0 is taken as 0, above 2 as 2. Off by default: on short titles a reach of 0 ranks as 1 does, and a search with typo matching is much cheaper, since with the exact rung counted no edit is searched for. A product search with few hits per query gains from 1.
Rest
bool
get/set
In, BM25F's best not counted below the counted set, up to the candidate bound: the rest, a fourth rung after two edits, in the last band, counted () and in the facets, so count, facets and list still agree. Off by default: the list is the counted set. On, it ranks as a relevance search does. Not read in.
Rerank
bool
get/set
Whether the candidates are reranked by Coverage (the field match level and the blend,), in either mode; on by default. Off:is BM25F alone, andorders inside a rung by BM25F's score.
Share
float
get/set
RerankHow Coverage and BM25F make one score:, both between 0 and 1; 0.2 by default. Coverage says how much of the query a document holds, letter for letter; BM25F knows the field weights, how rare a word is and how long a field is. Values outside 0..1 are taken as the nearest end. Not reranked (off) there is no Coverage, and BM25F alone orders.
TypingShift
float
get/set
How far the share moves towards Coverage alone while the last word is being typed:, 0.5 by default. 0 is no move, 1 is Coverage alone. A word the index does not hold whole ("warr"), or, in a query of one word, any word being typed ("war"). BM25F credits the letters typed wherever they stand, Coverage where a word starts, which is what is being typed.
LongFieldFrom
int
get/set
LongFieldWeightFloorThe text length, in characters, from which a field counts less in Coverage: where a record's field is longer, its Coverage weight there is multiplied by this length over its own to the power 0.18, never below; 735 characters count 0.79, 2 000 count 0.66. 200 by default; 0 is never.
LongFieldWeightFloor
float
get/set
LongFieldFromThe least a field's length may bring its Coverage weight down to (), as a share of its own, 0 by default: the damped fall seldom reaches a floor.
WholeQueryFloor
bool
get/set
The whole query as one string in a field puts a floor under the document's Coverage score, on by default.
FieldWeights
Dictionary<string, float>
get/set
Per field, its weight in Coverage in this search, in place of its own (): what a character confirmed there counts against the heaviest field. 0 or more; a name that is not a searchable field, or a weight below 0, makes the search return no records and say why in. Empty by default.
FieldBoosts
Dictionary<string, float>
get/set
Per field, a boost inside BM25F's saturation in this search (doubles the title's influence). Typical range 0.5 to 3; above about 5 a field overwhelms the IDF. Null or empty is no boosts. A field that is not searchable, or a boost below 0, is refused.
Boosts
Boost[]
get/set
The boosts of this search: a document a boost's filter holds is scored higher by its strength. Null or empty is none; there is no switch beside.
Buries
Bury[]
get/set
The buries of this search, the boosts' counterpart: a document a bury's filter holds is scored lower by its strength; nobody else moves. Null or empty is none.

Toleranceenum

The rungs the word lookup meets a query word on, from the nearest. A document is on its worst word's rung.

NameValue
Exact
0
OneEdit
1
TwoEdits
2
ThreeEdits
3

Rungsenum

RungsWhich of the word lookup's rungs a search counts (): the first that gives hits, or every one up to the tolerance.

NameValue
Fewest
0
All
1