Sentence Similarity
sentence-transformers
Safetensors
Japanese
luke
feature-extraction
yano0 commited on
Commit
69c2c00
·
verified ·
1 Parent(s): 83027ed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -3
README.md CHANGED
@@ -34,12 +34,18 @@ This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps
34
  ## Model Details
35
  The model is based on GLuCoSE and additional fine-tuned.
36
  Fine-tuning consists of the following steps.
37
- Step 1: Ensemble distillation
 
 
38
  - The embedded representation was distilled using E5-mistral, gte-Qwen2 and mE5-large as teacher models.
39
- Step 2: Contrast learning
 
 
40
  - Triples were created from JSNLI, MNLI, PAWS-X, JSeM and Mr.TyDi and used for training.
41
  - This training aimed to improve the overall performance as a sentence embedding model.
42
- Step 3: Search-specific contrastive learning.
 
 
43
  - In order to make the model more robust to the retrieval task, additional two-stage training with QA and question-answer data was conducted.
44
  - In the first stage, the synthetic dataset auto-wiki was used for training, while in the second stage, Japanese Wikipedia Human Retrieval, Mr.TyDi, MIRACL, JQaRA, MQA, Quiz Works and Quiz No Mori were used.
45
 
 
34
  ## Model Details
35
  The model is based on GLuCoSE and additional fine-tuned.
36
  Fine-tuning consists of the following steps.
37
+
38
+ **Step 1: Ensemble distillation**
39
+
40
  - The embedded representation was distilled using E5-mistral, gte-Qwen2 and mE5-large as teacher models.
41
+
42
+ **Step 2: Contrast learning**
43
+
44
  - Triples were created from JSNLI, MNLI, PAWS-X, JSeM and Mr.TyDi and used for training.
45
  - This training aimed to improve the overall performance as a sentence embedding model.
46
+
47
+ **Step 3: Search-specific contrastive learning.**
48
+
49
  - In order to make the model more robust to the retrieval task, additional two-stage training with QA and question-answer data was conducted.
50
  - In the first stage, the synthetic dataset auto-wiki was used for training, while in the second stage, Japanese Wikipedia Human Retrieval, Mr.TyDi, MIRACL, JQaRA, MQA, Quiz Works and Quiz No Mori were used.
51