SentenceTransformer based on sentence-transformers/paraphrase-multilingual-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-multilingual-mpnet-base-v2 on the multi_stsb_de, multi_stsb_es, multi_stsb_fr, multi_stsb_it, multi_stsb_nl, multi_stsb_pl, multi_stsb_pt, multi_stsb_ru and multi_stsb_zh datasets. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/paraphrase-multilingual-mpnet-base-v2
Maximum Sequence Length: 128 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Datasets:
Languages: de, en, es, fr, it, nl, pl, pt, ru, zh

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): MultiHeadGeneralizedPooling(
    (P): ModuleList(
      (0-7): 8 x Linear(in_features=768, out_features=96, bias=True)
    )
    (W1): ModuleList(
      (0-7): 8 x Linear(in_features=96, out_features=384, bias=True)
    )
    (W2): ModuleList(
      (0-7): 8 x Linear(in_features=384, out_features=96, bias=True)
    )
  )
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("RomainDarous/large_directFourEpoch_additivePooling_randomInit_stsModel")
# Run inference
sentences = [
    'Dois cães a lutar na neve.',
    'Dois cães brincam na neve.',
    'Pode sempre perguntar, então é a escolha do autor a aceitar ou não.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Semantic Similarity

Datasets: sts-eval, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test and sts-test
Evaluated with EmbeddingSimilarityEvaluator

Metric	sts-eval	sts-test
pearson_cosine	0.8344	0.7794
spearman_cosine	0.8514	0.7832

Semantic Similarity

Dataset: sts-eval
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.836
spearman_cosine	0.8555

Semantic Similarity

Dataset: sts-eval
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8334
spearman_cosine	0.8542

Semantic Similarity

Dataset: sts-eval
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8326
spearman_cosine	0.8526

Semantic Similarity

Dataset: sts-eval
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8325
spearman_cosine	0.8527

Semantic Similarity

Dataset: sts-eval
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8267
spearman_cosine	0.8466

Semantic Similarity

Dataset: sts-eval
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8326
spearman_cosine	0.854

Semantic Similarity

Dataset: sts-eval
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.827
spearman_cosine	0.8449

Semantic Similarity

Dataset: sts-eval
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8337
spearman_cosine	0.8512

Training Details

Training Datasets

multi_stsb_de

Dataset: multi_stsb_de at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 5 tokens
mean: 11.58 tokens
max: 37 tokens

min: 6 tokens
mean: 11.53 tokens
max: 36 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 5 tokens mean: 11.58 tokens max: 37 tokens	min: 6 tokens mean: 11.53 tokens max: 36 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`Ein Flugzeug hebt gerade ab.`	`Ein Flugzeug hebt gerade ab.`	`1.0`
`Ein Mann spielt eine große Flöte.`	`Ein Mann spielt eine Flöte.`	`0.7599999904632568`
`Ein Mann streicht geriebenen Käse auf eine Pizza.`	`Ein Mann streicht geriebenen Käse auf eine ungekochte Pizza.`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_es

Dataset: multi_stsb_es at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 7 tokens
mean: 12.21 tokens
max: 33 tokens

min: 7 tokens
mean: 12.07 tokens
max: 31 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 7 tokens mean: 12.21 tokens max: 33 tokens	min: 7 tokens mean: 12.07 tokens max: 31 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`Un avión está despegando.`	`Un avión está despegando.`	`1.0`
`Un hombre está tocando una gran flauta.`	`Un hombre está tocando una flauta.`	`0.7599999904632568`
`Un hombre está untando queso rallado en una pizza.`	`Un hombre está untando queso rallado en una pizza cruda.`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_fr

Dataset: multi_stsb_fr at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 6 tokens
mean: 12.6 tokens
max: 33 tokens

min: 6 tokens
mean: 12.49 tokens
max: 32 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 6 tokens mean: 12.6 tokens max: 33 tokens	min: 6 tokens mean: 12.49 tokens max: 32 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`Un avion est en train de décoller.`	`Un avion est en train de décoller.`	`1.0`
`Un homme joue d'une grande flûte.`	`Un homme joue de la flûte.`	`0.7599999904632568`
`Un homme étale du fromage râpé sur une pizza.`	`Un homme étale du fromage râpé sur une pizza non cuite.`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_it

Dataset: multi_stsb_it at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 7 tokens
mean: 12.77 tokens
max: 36 tokens

min: 8 tokens
mean: 12.69 tokens
max: 30 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 7 tokens mean: 12.77 tokens max: 36 tokens	min: 8 tokens mean: 12.69 tokens max: 30 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`Un aereo sta decollando.`	`Un aereo sta decollando.`	`1.0`
`Un uomo sta suonando un grande flauto.`	`Un uomo sta suonando un flauto.`	`0.7599999904632568`
`Un uomo sta spalmando del formaggio a pezzetti su una pizza.`	`Un uomo sta spalmando del formaggio a pezzetti su una pizza non cotta.`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_nl

Dataset: multi_stsb_nl at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 6 tokens
mean: 11.67 tokens
max: 33 tokens

min: 6 tokens
mean: 11.55 tokens
max: 29 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 6 tokens mean: 11.67 tokens max: 33 tokens	min: 6 tokens mean: 11.55 tokens max: 29 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`Er gaat een vliegtuig opstijgen.`	`Er gaat een vliegtuig opstijgen.`	`1.0`
`Een man speelt een grote fluit.`	`Een man speelt fluit.`	`0.7599999904632568`
`Een man smeert geraspte kaas op een pizza.`	`Een man strooit geraspte kaas op een ongekookte pizza.`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_pl

Dataset: multi_stsb_pl at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 5 tokens
mean: 12.2 tokens
max: 39 tokens

min: 5 tokens
mean: 12.11 tokens
max: 35 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 5 tokens mean: 12.2 tokens max: 39 tokens	min: 5 tokens mean: 12.11 tokens max: 35 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`Samolot wystartował.`	`Samolot wystartował.`	`1.0`
`Człowiek gra na dużym flecie.`	`Człowiek gra na flecie.`	`0.7599999904632568`
`Mężczyzna rozsiewa na pizzy rozdrobniony ser.`	`Mężczyzna rozsiewa rozdrobniony ser na niegotowanej pizzy.`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_pt

Dataset: multi_stsb_pt at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 7 tokens
mean: 12.33 tokens
max: 34 tokens

min: 7 tokens
mean: 12.29 tokens
max: 32 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 7 tokens mean: 12.33 tokens max: 34 tokens	min: 7 tokens mean: 12.29 tokens max: 32 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`Um avião está a descolar.`	`Um avião aéreo está a descolar.`	`1.0`
`Um homem está a tocar uma grande flauta.`	`Um homem está a tocar uma flauta.`	`0.7599999904632568`
`Um homem está a espalhar queijo desfiado numa pizza.`	`Um homem está a espalhar queijo desfiado sobre uma pizza não cozida.`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_ru

Dataset: multi_stsb_ru at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 5 tokens
mean: 11.19 tokens
max: 39 tokens

min: 5 tokens
mean: 11.17 tokens
max: 26 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 5 tokens mean: 11.19 tokens max: 39 tokens	min: 5 tokens mean: 11.17 tokens max: 26 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`Самолет взлетает.`	`Взлетает самолет.`	`1.0`
`Человек играет на большой флейте.`	`Человек играет на флейте.`	`0.7599999904632568`
`Мужчина разбрасывает сыр на пиццу.`	`Мужчина разбрасывает измельченный сыр на вареную пиццу.`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_zh

Dataset: multi_stsb_zh at 3acaa3d
Size: 5,749 training samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 6 tokens
mean: 10.7 tokens
max: 32 tokens

min: 7 tokens
mean: 10.79 tokens
max: 26 tokens

min: 0.0
mean: 0.45
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 6 tokens mean: 10.7 tokens max: 32 tokens	min: 7 tokens mean: 10.79 tokens max: 26 tokens	min: 0.0 mean: 0.45 max: 1.0

Samples:

sentence1	sentence2	score
`一架飞机正在起飞。`	`一架飞机正在起飞。`	`1.0`
`一个男人正在吹一支大笛子。`	`一个人在吹笛子。`	`0.7599999904632568`
`一名男子正在比萨饼上涂抹奶酪丝。`	`一名男子正在将奶酪丝涂抹在未熟的披萨上。`	`0.7599999904632568`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

Evaluation Datasets

multi_stsb_de

Dataset: multi_stsb_de at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 5 tokens
mean: 18.25 tokens
max: 47 tokens

min: 6 tokens
mean: 18.25 tokens
max: 54 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 5 tokens mean: 18.25 tokens max: 47 tokens	min: 6 tokens mean: 18.25 tokens max: 54 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`Ein Mann mit einem Schutzhelm tanzt.`	`Ein Mann mit einem Schutzhelm tanzt.`	`1.0`
`Ein kleines Kind reitet auf einem Pferd.`	`Ein Kind reitet auf einem Pferd.`	`0.949999988079071`
`Ein Mann verfüttert eine Maus an eine Schlange.`	`Der Mann füttert die Schlange mit einer Maus.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_es

Dataset: multi_stsb_es at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 7 tokens
mean: 17.98 tokens
max: 47 tokens

min: 7 tokens
mean: 17.86 tokens
max: 47 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 7 tokens mean: 17.98 tokens max: 47 tokens	min: 7 tokens mean: 17.86 tokens max: 47 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`Un hombre con un casco está bailando.`	`Un hombre con un casco está bailando.`	`1.0`
`Un niño pequeño está montando a caballo.`	`Un niño está montando a caballo.`	`0.949999988079071`
`Un hombre está alimentando a una serpiente con un ratón.`	`El hombre está alimentando a la serpiente con un ratón.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_fr

Dataset: multi_stsb_fr at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 6 tokens
mean: 19.7 tokens
max: 49 tokens

min: 6 tokens
mean: 19.65 tokens
max: 51 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 6 tokens mean: 19.7 tokens max: 49 tokens	min: 6 tokens mean: 19.65 tokens max: 51 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`Un homme avec un casque de sécurité est en train de danser.`	`Un homme portant un casque de sécurité est en train de danser.`	`1.0`
`Un jeune enfant monte à cheval.`	`Un enfant monte à cheval.`	`0.949999988079071`
`Un homme donne une souris à un serpent.`	`L'homme donne une souris au serpent.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_it

Dataset: multi_stsb_it at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 6 tokens
mean: 18.42 tokens
max: 46 tokens

min: 8 tokens
mean: 18.43 tokens
max: 53 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 6 tokens mean: 18.42 tokens max: 46 tokens	min: 8 tokens mean: 18.43 tokens max: 53 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`Un uomo con l'elmetto sta ballando.`	`Un uomo che indossa un elmetto sta ballando.`	`1.0`
`Un bambino piccolo sta cavalcando un cavallo.`	`Un bambino sta cavalcando un cavallo.`	`0.949999988079071`
`Un uomo sta dando da mangiare un topo a un serpente.`	`L'uomo sta dando da mangiare un topo al serpente.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_nl

Dataset: multi_stsb_nl at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 5 tokens
mean: 17.88 tokens
max: 50 tokens

min: 6 tokens
mean: 17.71 tokens
max: 51 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 5 tokens mean: 17.88 tokens max: 50 tokens	min: 6 tokens mean: 17.71 tokens max: 51 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`Een man met een helm is aan het dansen.`	`Een man met een helm is aan het dansen.`	`1.0`
`Een jong kind rijdt op een paard.`	`Een kind rijdt op een paard.`	`0.949999988079071`
`Een man voedt een muis aan een slang.`	`De man voert een muis aan de slang.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_pl

Dataset: multi_stsb_pl at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 6 tokens
mean: 18.54 tokens
max: 46 tokens

min: 6 tokens
mean: 18.43 tokens
max: 54 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 6 tokens mean: 18.54 tokens max: 46 tokens	min: 6 tokens mean: 18.43 tokens max: 54 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`Tańczy mężczyzna w twardym kapeluszu.`	`Tańczy mężczyzna w twardym kapeluszu.`	`1.0`
`Małe dziecko jedzie na koniu.`	`Dziecko jedzie na koniu.`	`0.949999988079071`
`Człowiek karmi węża myszką.`	`Ten człowiek karmi węża myszką.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_pt

Dataset: multi_stsb_pt at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 7 tokens
mean: 18.22 tokens
max: 46 tokens

min: 7 tokens
mean: 18.11 tokens
max: 46 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 7 tokens mean: 18.22 tokens max: 46 tokens	min: 7 tokens mean: 18.11 tokens max: 46 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`Um homem de chapéu duro está a dançar.`	`Um homem com um capacete está a dançar.`	`1.0`
`Uma criança pequena está a montar a cavalo.`	`Uma criança está a montar a cavalo.`	`0.949999988079071`
`Um homem está a alimentar um rato a uma cobra.`	`O homem está a alimentar a cobra com um rato.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_ru

Dataset: multi_stsb_ru at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 6 tokens
mean: 17.92 tokens
max: 49 tokens

min: 5 tokens
mean: 17.75 tokens
max: 47 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 6 tokens mean: 17.92 tokens max: 49 tokens	min: 5 tokens mean: 17.75 tokens max: 47 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`Человек в твердой шляпе танцует.`	`Мужчина в твердой шляпе танцует.`	`1.0`
`Маленький ребенок едет верхом на лошади.`	`Ребенок едет на лошади.`	`0.949999988079071`
`Мужчина кормит мышь змее.`	`Человек кормит змею мышью.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

multi_stsb_zh

Dataset: multi_stsb_zh at 3acaa3d
Size: 1,500 evaluation samples
Columns: sentence1, sentence2, and score
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 score
type string string float
details
min: 6 tokens
mean: 15.37 tokens
max: 46 tokens

min: 5 tokens
mean: 15.24 tokens
max: 46 tokens

min: 0.0
mean: 0.42
max: 1.0

	sentence1	sentence2	score
type	string	string	float
details	min: 6 tokens mean: 15.37 tokens max: 46 tokens	min: 5 tokens mean: 15.24 tokens max: 46 tokens	min: 0.0 mean: 0.42 max: 1.0

Samples:

sentence1	sentence2	score
`一个戴着硬帽子的人在跳舞。`	`一个戴着硬帽的人在跳舞。`	`1.0`
`一个小孩子在骑马。`	`一个孩子在骑马。`	`0.949999988079071`
`一个人正在用老鼠喂蛇。`	`那人正在给蛇喂老鼠。`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
num_train_epochs: 4
warmup_ratio: 0.1

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 4
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	multi stsb de loss	multi stsb es loss	multi stsb fr loss	multi stsb it loss	multi stsb nl loss	multi stsb pl loss	multi stsb pt loss	multi stsb ru loss	multi stsb zh loss	sts-eval_spearman_cosine	sts-test_spearman_cosine
1.0	3240	4.3573	4.3951	4.4099	4.4457	4.4373	4.4244	4.5118	4.4754	4.4640	4.3865	0.8335	-
2.0	6480	3.9482	4.9132	4.9378	5.0707	4.9663	4.8902	4.9785	4.9303	5.0369	4.8343	0.8459	-
3.0	9720	3.6251	5.7658	5.6844	5.8164	5.8314	5.7706	5.6396	5.8485	5.8846	5.4936	0.8549	-
4.0	12960	3.2761	6.6703	6.6595	6.8158	6.8143	6.6985	6.6876	6.7388	6.9709	6.4154	0.8512	-
-1	-1	-	-	-	-	-	-	-	-	-	-	-	0.7832

Framework Versions

Python: 3.10.13
Sentence Transformers: 3.4.1
Transformers: 4.48.2
PyTorch: 2.1.2+cu121
Accelerate: 1.3.0
Datasets: 2.16.1
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}

Downloads last month: 5

Safetensors

Model size

0.3B params

Tensor type

F32

Model tree for RomainDarous/large_directFourEpoch_additivePooling_randomInit_stsModel

Base model

sentence-transformers/paraphrase-multilingual-mpnet-base-v2

Finetuned

(133)

this model

Dataset used to train RomainDarous/large_directFourEpoch_additivePooling_randomInit_stsModel

Paper for RomainDarous/large_directFourEpoch_additivePooling_randomInit_stsModel

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Paper • 1908.10084 • Published Aug 27, 2019 • 9

Evaluation results

Pearson Cosine on sts eval
self-reported

0.834
Spearman Cosine on sts eval
self-reported

0.851
Pearson Cosine on sts eval
self-reported

0.836
Spearman Cosine on sts eval
self-reported

0.855
Pearson Cosine on sts eval
self-reported

0.833
Spearman Cosine on sts eval
self-reported

0.854
Pearson Cosine on sts eval
self-reported

0.833
Spearman Cosine on sts eval
self-reported

0.853
Pearson Cosine on sts eval
self-reported

0.833
Spearman Cosine on sts eval
self-reported

0.853