s

Mmlw Roberta Base

Nov 2023
sMmlw Roberta Base
Compare in
on
API price not publicprice
40.2MTEB English v2
Nov 2023released
768Dimensions
1K tokensMax input
PolishLanguages

Rankingconfiguration unstated?

Full ranking →
1Inf Retriever v168.82Gte Qwen2 7B Instruct68.13BOOM 4B v167.94SFR Embedding Mistral67.85Gte Qwen2 1.5B Instruct67.6
···
73Sentence Croissant Alpha v0.342.374Mmlw Roberta Base42.275Snowflake Arctic Embed M Long42.1

Strong at

C
Embeddings & Retrieval#147 of 200

Alternatives?

ModelScoreCostPriceWhat changes
iBetterMultilingual E5 Large InstructunstatedOpen+44%—$0.01Memory (4-bit): 0.1 GB → 0.3 GBiOpenInf Retriever v1unstated+63%——Memory (4-bit): 0.1 GB → 4.2 GBtBetterGte LargeunstatedOpen+8%—$0.01Memory (4-bit): 0.1 GB → 0.2 GBiBetterE5 Large v2unstatedOpen+6%—$0.01Memory (4-bit): 0.1 GB → 0.2 GBBetterText Embedding 3 Smallunstated+40%—$0.02Languages: not verified

Run it yourself

estimate
Fits on✓ 8 GB (phone, laptop) · full quality✓ 16 GB · full quality✓ 24 GB GPU · full quality✓ 48 GB GPU · full quality✓ 80 GB GPU · full quality
LicenceApache-2.0free for commercial use

Benchmarks

in Embeddings & Retrieval
BEIR · Mean over tasks · English39.2#148
BEIR · Mean over task types · English39.2#148
BEIR · Retrieval · English39.2#148
MTEB Persian v2 · Reranking · Persian18.2#146
JMTEB v2 · Clustering · Japanese13.8#126
JMTEB v2 · STS · Japanese4.1#167
LongEmbed · Mean over tasks · English33.2#112
LongEmbed · Mean over task types · English33.2#112
LongEmbed · Retrieval · English33.2#112
MTEB German v1 · Mean over tasks · German37.7#77
MTEB German v1 · Mean over task types · German35.4#81
MTEB German v1 · Classification · German51.1#89
MTEB German v1 · Pair Classification · German49.2#122
MTEB German v1 · STS · German45.8#112
MTEB German v1 · Clustering · German23.7#97
MTEB German v1 · Retrieval · German27.9#94
MTEB German v1 · Reranking · German14.7#124
MTEB Europe v1 · Mean over tasks · multilingual (29 languages)46.2#48
MTEB Europe v1 · Mean over task types · multilingual (29 languages)40.1#49
MTEB Europe v1 · Classification · multilingual (23 languages)53.2#55
MTEB Europe v1 · Instruction Reranking · English-3.4#183
MTEB Europe v1 · Multilabel Classification · multilingual (23 languages)11.2#116
MTEB Europe v1 · STS · multilingual (8 languages)68.7#53
MTEB Europe v1 · Pair Classification · multilingual (11 languages)84.5#75
MTEB Europe v1 · Clustering · multilingual (29 languages)36.4#95
MTEB Europe v1 · Retrieval · multilingual (25 languages)34.4#116
MTEB Europe v1 · Reranking · multilingual (12 languages)43.5#140
MTEB Europe v1 · Bitext Mining · multilingual (28 languages)32.9#137
MTEB French v1 · Mean over tasks · multilingual (2 languages)44.4#74
MTEB French v1 · Mean over task types · multilingual (2 languages)45.9#75
MTEB French v1 · STS · French67.4#77
MTEB French v1 · Retrieval · multilingual (2 languages)19.9#116
MTEB French v1 · Classification · French54.8#78
MTEB French v1 · Clustering · French41.2#56
MTEB French v1 · Reranking · French54.1#114
MTEB French v1 · Pair Classification · French53.5#167
MTEB French v1 · Summarization · French30.7#46
MTEB Indic v1 · Mean over tasks · multilingual (25 languages)31#119
MTEB Indic v1 · Mean over task types · multilingual (25 languages)21.3#113
MTEB Indic v1 · Classification · multilingual (8 languages)41.6#127
MTEB Indic v1 · Bitext Mining · multilingual (20 languages)4.6#105
MTEB Indic v1 · STS · multilingual (13 languages)7.8#112
MTEB Indic v1 · Reranking · multilingual (2 languages)33#183
MTEB Indic v1 · Retrieval · multilingual (14 languages)7.9#136
MTEB Indic v1 · Pair Classification · Hindi50.4#180
MTEB Indic v1 · Clustering · multilingual (21 languages)3.9#126
MTEB Korean v2 · STS · Korean6#98
MTEB Korean v2 · Classification · Korean21.6#114
MTEB Korean v2 · Reranking · Korean5.5#138
MTEB Dutch v1 · Mean over tasks · Dutch36.4#100
MTEB Dutch v1 · Mean over task types · Dutch44.8#98
MTEB Dutch v1 · Retrieval · Dutch24.9#106
MTEB Dutch v1 · Classification · Dutch49.1#81
MTEB Dutch v1 · Multilabel Classification · Dutch29.3#96
MTEB Dutch v1 · STS · Dutch56.5#106
MTEB Dutch v1 · Clustering · Dutch22.1#92
MTEB Dutch v1 · Reranking · Dutch72.5#170
MTEB Dutch v1 · Pair Classification · Dutch59.5#104
MTEB Polish v1 · Classification · Polish62.9#30
MTEB Polish v1 · STS · Polish70.7#7
MTEB Polish v1 · Pair Classification · Polish88.2#5
MTEB Portuguese v1 · Multilabel Classification · Portuguese16.7#171
MTEB Law v1 · Mean over tasks · multilingual (3 languages)35.3#125
MTEB Law v1 · Mean over task types · multilingual (3 languages)35.3#125
MTEB Law v1 · Retrieval · multilingual (3 languages)35.3#125
MTEB Medical v1 · Mean over tasks · multilingual (10 languages)38.6#75
MTEB Medical v1 · Mean over task types · multilingual (10 languages)29.6#97
MTEB Medical v1 · Retrieval · multilingual (10 languages)42.2#74
MTEB Medical v1 · Clustering · English36.9#74
MTEB Medical v1 · Reranking · Mandarin Chinese9.5#192
FollowIR · Mean over tasks · English-3.4#183
FollowIR · Mean over task types · English-3.4#183
FollowIR · Instruction Reranking · English-3.4#183
MTEB Spanish v1 · STS · multilingual (2 languages)61.4#63
MTEB Spanish v1 · Clustering · Spanish38.2#92
MTEB Spanish v1 · Pair Classification · Spanish58.7#124
MTEB Spanish v1 · Reranking · Spanish16.8#131
MTEB Thai v1 · Pair Classification · Thai52.1#123
MTEB Thai v1 · Clustering · Thai4#134
MTEB Scandinavian v1 · Classification · multilingual (6 languages)48.3#112
MTEB Scandinavian v1 · Clustering · multilingual (2 languages)31.7#118
MTEB Scandinavian v1 · Bitext Mining · multilingual (3 languages)60.9#134
MTEB Russian v1.1 · Mean over tasks · Russian17.5#106
MTEB Russian v1.1 · Mean over task types · Russian18.6#106
MTEB Russian v1.1 · Classification · Russian22.8#145
MTEB Russian v1.1 · Clustering · Russian13.7#146
MTEB Russian v1.1 · Reranking · Russian9#153
MTEB Russian v1.1 · Multilabel Classification · Russian25.7#136
MTEB Russian v1.1 · STS · Russian13.3#139
MTEB Russian v1.1 · Retrieval · Russian0.5#107
MTEB Russian v1.1 · Pair Classification · Russian45.2#186
RTEB French beta · Mean over tasks · multilingual (2 languages)4.1#185
RTEB German beta · Mean over tasks · German10.1#188
RTEB Law beta · Mean over tasks · multilingual (4 languages)27.4#171
MTEB multilingual v2 · Mean over tasks · multilingual (1037 languages)41.5#80
MTEB multilingual v2 · Mean over task types · multilingual (1037 languages)34.7#80
MTEB multilingual v2 · Clustering · multilingual (208 languages)40.2#98
MTEB multilingual v2 · STS · multilingual (35 languages)51.7#104
MTEB multilingual v2 · Classification · multilingual (119 languages)47.4#113
MTEB multilingual v2 · Retrieval · multilingual (120 languages)30.4#148
MTEB multilingual v2 · Multilabel Classification · multilingual (25 languages)15#137
MTEB multilingual v2 · Bitext Mining · multilingual (1026 languages)23.4#130
MTEB multilingual v2 · Instruction Reranking · English-3.4#183
MTEB multilingual v2 · Reranking · multilingual (22 languages)34.4#164
MTEB multilingual v2 · Pair Classification · multilingual (24 languages)73.2#96

FAQ

How much does Mmlw Roberta Base cost?›

There is no per-unit API price: API price not public.

Is Mmlw Roberta Base free to use?›

There is no free API. The weights are open, so you can run it yourself.

How good is Mmlw Roberta Base, and what is it best at?›

#147 of 200 in Embeddings.

Is Mmlw Roberta Base better than Sentence Croissant Alpha v0.3?›

On MTEB Medical v1 · Retrieval · multilingual (10 languages), Mmlw Roberta Base scores 42.2 vs 42.3 — about the same.

When was Mmlw Roberta Base released? Is there a newer version?›

Released 2023-11-17. It is the current version in its line.

Is Mmlw Roberta Base open source?›

Yes, the weights are open (Apache-2.0 licence).

Can I run Mmlw Roberta Base locally?›

The weights are open and can be self-hosted; hardware needs are not confirmed yet.

What is Mmlw Roberta Base?›

Mmlw Roberta Base is an open-weights embeddings & retrieval model from sdadas, released Nov 2023.

What are the best alternatives to Mmlw Roberta Base?›

Multilingual E5 Large Instruct (better), Inf Retriever v1 (open weights).