M

Wartortle

Apr 2024
MWartortle
Compare in
on
self-host onlyprice
26.9MTEB English v2
Apr 2024released
384Dimensions
1K tokensMax input

Rankingconfiguration unstated?

Full ranking →
1Multilingual E5 Base332Rubert Tiny232.53Sentence Croissant Alpha v0.332.54Speed Embedding 7B Instruct32.45Bilingual Embedding Small32.3
···
46Mmlw Roberta Base30.748Wartortle30.749Bge Base En v1.530.7

Strong at

C
Embeddings & Retrieval#168 of 200

Run it yourself

estimate
Fits on✓ 8 GB (phone, laptop) · full quality✓ 16 GB · full quality✓ 24 GB GPU · full quality✓ 48 GB GPU · full quality✓ 80 GB GPU · full quality
LicenceMITfree for commercial use

Benchmarks

in Embeddings & Retrieval
BEIR · Mean over tasks · English29.3#167
BEIR · Mean over task types · English29.3#167
BEIR · Retrieval · English29.3#167
MTEB Persian v2 · Reranking · Persian30.3#109
JMTEB v2 · Clustering · Japanese12.9#131
JMTEB v2 · STS · Japanese58.2#118
LongEmbed · Mean over tasks · English31.1#121
LongEmbed · Mean over task types · English31.1#121
LongEmbed · Retrieval · English31.1#121
MTEB German v1 · Mean over tasks · German35.3#90
MTEB German v1 · Mean over task types · German33.6#91
MTEB German v1 · Classification · German50.8#91
MTEB German v1 · Pair Classification · German48.1#165
MTEB German v1 · STS · German47.5#101
MTEB German v1 · Clustering · German16.8#123
MTEB German v1 · Retrieval · German23.2#102
MTEB German v1 · Reranking · German15.4#122
MTEB Europe v1 · Classification · multilingual (23 languages)49.5#98
MTEB Europe v1 · Instruction Reranking · English-1.5#106
MTEB Europe v1 · Multilabel Classification · multilingual (23 languages)8.5#135
MTEB Europe v1 · Pair Classification · multilingual (11 languages)79.6#133
MTEB Europe v1 · Clustering · multilingual (29 languages)29.4#142
MTEB Europe v1 · Retrieval · multilingual (25 languages)30.6#125
MTEB Europe v1 · Reranking · multilingual (12 languages)42.2#148
MTEB Europe v1 · Bitext Mining · multilingual (28 languages)26.4#153
MTEB French v1 · Mean over tasks · multilingual (2 languages)41.8#80
MTEB French v1 · Mean over task types · multilingual (2 languages)44.1#80
MTEB French v1 · STS · French66.1#80
MTEB French v1 · Retrieval · multilingual (2 languages)24.7#108
MTEB French v1 · Classification · French50.8#99
MTEB French v1 · Clustering · French33.3#95
MTEB French v1 · Reranking · French50.7#118
MTEB French v1 · Pair Classification · French52#173
MTEB French v1 · Summarization · French30.7#48
MTEB Indic v1 · Mean over tasks · multilingual (25 languages)32.1#99
MTEB Indic v1 · Mean over task types · multilingual (25 languages)23#94
MTEB Indic v1 · Classification · multilingual (8 languages)42.4#110
MTEB Indic v1 · Bitext Mining · multilingual (20 languages)2.3#144
MTEB Indic v1 · STS · multilingual (13 languages)2.7#148
MTEB Indic v1 · Reranking · multilingual (2 languages)47#138
MTEB Indic v1 · Retrieval · multilingual (14 languages)12.7#115
MTEB Indic v1 · Pair Classification · Hindi51.7#163
MTEB Indic v1 · Clustering · multilingual (21 languages)2.6#182
MTEB Korean v2 · Classification · Korean23.2#85
MTEB Korean v2 · Reranking · Korean13.2#95
MTEB Dutch v1 · Mean over tasks · Dutch33.3#108
MTEB Dutch v1 · Mean over task types · Dutch43.4#107
MTEB Dutch v1 · Retrieval · Dutch24.2#109
MTEB Dutch v1 · Classification · Dutch44.2#108
MTEB Dutch v1 · Multilabel Classification · Dutch24.8#114
MTEB Dutch v1 · STS · Dutch58.6#95
MTEB Dutch v1 · Clustering · Dutch15.1#113
MTEB Dutch v1 · Reranking · Dutch76.2#149
MTEB Dutch v1 · Pair Classification · Dutch61.1#96
MTEB Polish v1 · Mean over tasks · Polish45.1#73
MTEB Polish v1 · Mean over task types · Polish44.9#71
MTEB Polish v1 · Classification · Polish41.2#102
MTEB Polish v1 · Clustering · Polish20#108
MTEB Polish v1 · STS · Polish55.2#87
MTEB Polish v1 · Pair Classification · Polish63#116
MTEB Portuguese v1 · Multilabel Classification · Portuguese15.8#179
MTEB Law v1 · Mean over tasks · multilingual (3 languages)33#134
MTEB Law v1 · Mean over task types · multilingual (3 languages)33#134
MTEB Law v1 · Retrieval · multilingual (3 languages)33#134
MTEB Medical v1 · Mean over tasks · multilingual (10 languages)25.2#114
MTEB Medical v1 · Mean over task types · multilingual (10 languages)24.6#108
MTEB Medical v1 · Retrieval · multilingual (10 languages)24.7#116
MTEB Medical v1 · Clustering · English31.1#187
MTEB Medical v1 · Reranking · Mandarin Chinese18#154
FollowIR · Mean over tasks · English-1.5#106
FollowIR · Mean over task types · English-1.5#106
FollowIR · Instruction Reranking · English-1.5#106
MTEB Spanish v1 · Clustering · Spanish34.9#108
MTEB Spanish v1 · Pair Classification · Spanish57.1#137
MTEB Spanish v1 · Reranking · Spanish18.8#125
MTEB Spanish v1 · Classification · Spanish66.4#82
MTEB Thai v1 · Pair Classification · Thai50.9#177
MTEB Thai v1 · Clustering · Thai3.6#155
MTEB Scandinavian v1 · Mean over tasks · multilingual (6 languages)40.2#98
MTEB Scandinavian v1 · Mean over task types · multilingual (6 languages)42#94
MTEB Scandinavian v1 · Classification · multilingual (6 languages)48.5#107
MTEB Scandinavian v1 · Retrieval · multilingual (3 languages)31#105
MTEB Scandinavian v1 · Clustering · multilingual (2 languages)25.4#145
MTEB Scandinavian v1 · Bitext Mining · multilingual (3 languages)63.2#105
MTEB Russian v1.1 · Mean over tasks · Russian23#90
MTEB Russian v1.1 · Mean over task types · Russian23.7#90
MTEB Russian v1.1 · Classification · Russian27.4#133
MTEB Russian v1.1 · Clustering · Russian12.9#152
MTEB Russian v1.1 · Reranking · Russian17.2#127
MTEB Russian v1.1 · Multilabel Classification · Russian25.8#134
MTEB Russian v1.1 · STS · Russian36.6#116
MTEB Russian v1.1 · Retrieval · Russian0.6#103
MTEB Russian v1.1 · Pair Classification · Russian45.7#182
RTEB French beta · Mean over tasks · multilingual (2 languages)6#178
RTEB German beta · Mean over tasks · German14.5#161
RTEB Law beta · Mean over tasks · multilingual (4 languages)28#167
MTEB multilingual v2 · Clustering · multilingual (208 languages)34.6#134
MTEB multilingual v2 · Classification · multilingual (119 languages)46.6#125
MTEB multilingual v2 · Retrieval · multilingual (120 languages)27.2#153
MTEB multilingual v2 · Multilabel Classification · multilingual (25 languages)14#150
MTEB multilingual v2 · Bitext Mining · multilingual (1026 languages)19.7#154
MTEB multilingual v2 · Instruction Reranking · English-1.5#106
MTEB multilingual v2 · Reranking · multilingual (22 languages)39.6#141
MTEB multilingual v2 · Pair Classification · multilingual (24 languages)71.2#149

FAQ

How much does Wartortle cost?›

There is no per-unit API price: self-host only.

Is Wartortle free to use?›

There is no free API. The weights are open, so you can run it yourself.

How good is Wartortle, and what is it best at?›

#168 of 200 in Embeddings.

Is Wartortle better than Mmlw Roberta Base?›

On MTEB French v1 · Summarization · French, Wartortle scores 30.7 vs 30.7 — about the same.

When was Wartortle released? Is there a newer version?›

Released 2024-04-30. It is the current version in its line.

Is Wartortle open source?›

Yes, the weights are open (MIT licence).

Can I run Wartortle locally?›

The weights are open and can be self-hosted; hardware needs are not confirmed yet.

What is Wartortle?›

Wartortle is an open-weights embeddings & retrieval model from Mihaiii, released Apr 2024.

What are the best alternatives to Wartortle?›

Multilingual E5 Base (better).