Hugging Face's logo

codersan
/
eFuck

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:131157
loss:MultipleNegativesRankingLoss
text-embeddings-inference
Model card Files
xet
Community

SentenceTransformer based on intfloat/multilingual-e5-small

This is a sentence-transformers model finetuned from intfloat/multilingual-e5-small. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: intfloat/multilingual-e5-small
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 384 dimensions
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("codersan/eFuck")
# Run inference
sentences = [
    'برای تبدیل شدن به نویسنده برتر Quora ، چند بازدید و پاسخ لازم است؟',
    'چگونه می توانم نویسنده برتر Quora شوم ، از صعود بیشتر و آمار بهتر استفاده کنم؟',
    'من به دنبال خرید دوچرخه جدید هستم.Suzuki Gixxer 155 یا Honda Hornet 160r.کدام یک را بخرید؟',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

  • Size: 131,157 training samples
  • Columns: anchor and positive
  • Approximate statistics based on the first 1000 samples:
    anchor positive
    type string string
    details
    • min: 6 tokens
    • mean: 16.81 tokens
    • max: 97 tokens
    • min: 5 tokens
    • mean: 16.59 tokens
    • max: 59 tokens
  • Samples:
    anchor positive
    وقتی سوال من به عنوان "این سوال ممکن است به ویرایش نیاز داشته باشد" چه کاری باید انجام دهم ، اما نمی توانم دلیل آن را پیدا کنم؟ چرا سوال من به عنوان نیاز به پیشرفت مشخص شده است؟
    چگونه می توانید یک فایل رمزگذاری شده را با دانستن اینکه این یک فایل تصویری است بدون دانستن گسترش پرونده یا کلید ، رمزگشایی کنید؟ چگونه می توانید یک فایل رمزگذاری شده را رمزگشایی کنید و بدانید که این یک فایل تصویری است بدون اینکه از پسوند پرونده اطلاع داشته باشید؟
    احساس می کنم خودکشی می کنم ، چگونه باید با آن برخورد کنم؟ احساس می کنم خودکشی می کنم.چه کاری باید انجام دهم؟
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim"
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 32
  • learning_rate: 2e-05
  • weight_decay: 0.005
  • num_train_epochs: 10
  • warmup_ratio: 0.1
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: no
  • prediction_loss_only: True
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 8
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.005
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1
  • num_train_epochs: 10
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • eval_use_gather_object: False
  • average_tokens_across_devices: False
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional

Training Logs

Click to expand
Epoch Step Training Loss
0.0244 100 1.3984
0.0488 200 0.8762
0.0732 300 0.2492
0.0976 400 0.0754
0.1220 500 0.0809
0.1464 600 0.0789
0.1708 700 0.076
0.1952 800 0.0642
0.2196 900 0.0743
0.2440 1000 0.0605
0.2684 1100 0.0705
0.2928 1200 0.0594
0.3172 1300 0.0565
0.3415 1400 0.071
0.3659 1500 0.0476
0.3903 1600 0.0514
0.4147 1700 0.0584
0.4391 1800 0.0649
0.4635 1900 0.0485
0.4879 2000 0.0556
0.5123 2100 0.0594
0.5367 2200 0.0556
0.5611 2300 0.0439
0.5855 2400 0.0619
0.6099 2500 0.0553
0.6343 2600 0.0393
0.6587 2700 0.0458
0.6831 2800 0.0476
0.7075 2900 0.0535
0.7319 3000 0.0439
0.7563 3100 0.0438
0.7807 3200 0.052
0.8051 3300 0.0514
0.8295 3400 0.0549
0.8539 3500 0.0439
0.8783 3600 0.0429
0.9027 3700 0.0442
0.9271 3800 0.0643
0.9515 3900 0.0408
0.9758 4000 0.0403
1.0002 4100 0.0446
1.0246 4200 0.0527
1.0490 4300 0.0545
1.0734 4400 0.0517
1.0978 4500 0.0299
1.1222 4600 0.0444
1.1466 4700 0.0475
1.1710 4800 0.0414
1.1954 4900 0.0386
1.2198 5000 0.0508
1.2442 5100 0.0384
1.2686 5200 0.0453
1.2930 5300 0.0401
1.3174 5400 0.0328
1.3418 5500 0.0456
1.3662 5600 0.0295
1.3906 5700 0.0366
1.4150 5800 0.0431
1.4394 5900 0.0442
1.4638 6000 0.0343
1.4882 6100 0.0405
1.5126 6200 0.0357
1.5370 6300 0.0423
1.5614 6400 0.0288
1.5858 6500 0.0393
1.6101 6600 0.0369
1.6345 6700 0.0245
1.6589 6800 0.0286
1.6833 6900 0.0325
1.7077 7000 0.0311
1.7321 7100 0.0272
1.7565 7200 0.0261
1.7809 7300 0.0296
1.8053 7400 0.0343
1.8297 7500 0.036
1.8541 7600 0.0225
1.8785 7700 0.0232
1.9029 7800 0.0275
1.9273 7900 0.0394
1.9517 8000 0.0297
1.9761 8100 0.0249
2.0005 8200 0.0268
2.0249 8300 0.0269
2.0493 8400 0.0296
2.0737 8500 0.0326
2.0981 8600 0.0183
2.1225 8700 0.024
2.1469 8800 0.0298
2.1713 8900 0.0273
2.1957 9000 0.0244
2.2201 9100 0.0308
2.2444 9200 0.0247
2.2688 9300 0.0299
2.2932 9400 0.0222
2.3176 9500 0.0213
2.3420 9600 0.0316
2.3664 9700 0.0157
2.3908 9800 0.0248
2.4152 9900 0.028
2.4396 10000 0.0269
2.4640 10100 0.0214
2.4884 10200 0.0242
2.5128 10300 0.0222
2.5372 10400 0.0253
2.5616 10500 0.0175
2.5860 10600 0.0269
2.6104 10700 0.0281
2.6348 10800 0.014
2.6592 10900 0.0187
2.6836 11000 0.0204
2.7080 11100 0.0228
2.7324 11200 0.0193
2.7568 11300 0.014
2.7812 11400 0.0171
2.8056 11500 0.0213
2.8300 11600 0.025
2.8544 11700 0.0138
2.8788 11800 0.0133
2.9031 11900 0.021
2.9275 12000 0.0256
2.9519 12100 0.019
2.9763 12200 0.0149
3.0007 12300 0.0192
3.0251 12400 0.0194
3.0495 12500 0.0179
3.0739 12600 0.0218
3.0983 12700 0.0126
3.1227 12800 0.018
3.1471 12900 0.0188
3.1715 13000 0.0181
3.1959 13100 0.0186
3.2203 13200 0.0235
3.2447 13300 0.0172
3.2691 13400 0.0183
3.2935 13500 0.0155
3.3179 13600 0.0135
3.3423 13700 0.0236
3.3667 13800 0.0115
3.3911 13900 0.0162
3.4155 14000 0.0207
3.4399 14100 0.0174
3.4643 14200 0.0128
3.4887 14300 0.0202
3.5131 14400 0.0165
3.5374 14500 0.0162
3.5618 14600 0.015
3.5862 14700 0.0203
3.6106 14800 0.0222
3.6350 14900 0.0105
3.6594 15000 0.014
3.6838 15100 0.0146
3.7082 15200 0.015
3.7326 15300 0.0153
3.7570 15400 0.0099
3.7814 15500 0.0105
3.8058 15600 0.0168
3.8302 15700 0.0185
3.8546 15800 0.0104
3.8790 15900 0.01
3.9034 16000 0.0142
3.9278 16100 0.0197
3.9522 16200 0.013
3.9766 16300 0.0137
4.0010 16400 0.0133
4.0254 16500 0.0132
4.0498 16600 0.0124
4.0742 16700 0.0141
4.0986 16800 0.0099
4.1230 16900 0.0113
4.1474 17000 0.0149
4.1717 17100 0.0145
4.1961 17200 0.0129
4.2205 17300 0.0185
4.2449 17400 0.0138
4.2693 17500 0.0133
4.2937 17600 0.0107
4.3181 17700 0.0092
4.3425 17800 0.0175
4.3669 17900 0.0097
4.3913 18000 0.0111
4.4157 18100 0.0136
4.4401 18200 0.0122
4.4645 18300 0.0095
4.4889 18400 0.0141
4.5133 18500 0.0094
4.5377 18600 0.0123
4.5621 18700 0.0108
4.5865 18800 0.0145
4.6109 18900 0.0195
4.6353 19000 0.0099
4.6597 19100 0.0107
4.6841 19200 0.0105
4.7085 19300 0.0124
4.7329 19400 0.012
4.7573 19500 0.0081
4.7817 19600 0.0081
4.8061 19700 0.0111
4.8304 19800 0.0141
4.8548 19900 0.0073
4.8792 20000 0.0094
4.9036 20100 0.011
4.9280 20200 0.0157
4.9524 20300 0.0086
4.9768 20400 0.0093
5.0012 20500 0.011
5.0256 20600 0.0107
5.0500 20700 0.0094
5.0744 20800 0.008
5.0988 20900 0.0076
5.1232 21000 0.0088
5.1476 21100 0.0119
5.1720 21200 0.0118
5.1964 21300 0.0105
5.2208 21400 0.0138
5.2452 21500 0.0109
5.2696 21600 0.0101
5.2940 21700 0.008
5.3184 21800 0.0068
5.3428 21900 0.0123
5.3672 22000 0.0086
5.3916 22100 0.0084
5.4160 22200 0.0113
5.4404 22300 0.0086
5.4647 22400 0.0076
5.4891 22500 0.0101
5.5135 22600 0.0083
5.5379 22700 0.0116
5.5623 22800 0.0083
5.5867 22900 0.0137
5.6111 23000 0.0144
5.6355 23100 0.0081
5.6599 23200 0.006
5.6843 23300 0.0096
5.7087 23400 0.0098
5.7331 23500 0.0096
5.7575 23600 0.0063
5.7819 23700 0.0052
5.8063 23800 0.008
5.8307 23900 0.0117
5.8551 24000 0.0053
5.8795 24100 0.0077
5.9039 24200 0.0086
5.9283 24300 0.0129
5.9527 24400 0.0085
5.9771 24500 0.0064
6.0015 24600 0.0092
6.0259 24700 0.0076
6.0503 24800 0.0078
6.0747 24900 0.0074
6.0990 25000 0.0064
6.1234 25100 0.0067
6.1478 25200 0.0091
6.1722 25300 0.0087
6.1966 25400 0.0076
6.2210 25500 0.0104
6.2454 25600 0.0077
6.2698 25700 0.0074
6.2942 25800 0.0055
6.3186 25900 0.0059
6.3430 26000 0.0092
6.3674 26100 0.0051
6.3918 26200 0.0075
6.4162 26300 0.0093
6.4406 26400 0.0073
6.4650 26500 0.0051
6.4894 26600 0.0093
6.5138 26700 0.0065
6.5382 26800 0.0072
6.5626 26900 0.0075
6.5870 27000 0.0111
6.6114 27100 0.0139
6.6358 27200 0.0066
6.6602 27300 0.0062
6.6846 27400 0.0078
6.7090 27500 0.0084
6.7333 27600 0.0077
6.7577 27700 0.0055
6.7821 27800 0.0039
6.8065 27900 0.0082
6.8309 28000 0.0101
6.8553 28100 0.0041
6.8797 28200 0.0058
6.9041 28300 0.0058
6.9285 28400 0.0109
6.9529 28500 0.0054
6.9773 28600 0.0061
7.0017 28700 0.0078
7.0261 28800 0.0065
7.0505 28900 0.0061
7.0749 29000 0.0049
7.0993 29100 0.0062
7.1237 29200 0.0052
7.1481 29300 0.0073
7.1725 29400 0.0072
7.1969 29500 0.0067
7.2213 29600 0.0093
7.2457 29700 0.008
7.2701 29800 0.0057
7.2945 29900 0.0051
7.3189 30000 0.0046
7.3433 30100 0.0078
7.3677 30200 0.0041
7.3920 30300 0.0054
7.4164 30400 0.008
7.4408 30500 0.0056
7.4652 30600 0.0037
7.4896 30700 0.0071
7.5140 30800 0.0058
7.5384 30900 0.0074
7.5628 31000 0.0059
7.5872 31100 0.0088
7.6116 31200 0.0102
7.6360 31300 0.0058
7.6604 31400 0.0044
7.6848 31500 0.0065
7.7092 31600 0.007
7.7336 31700 0.0078
7.7580 31800 0.0048
7.7824 31900 0.0033
7.8068 32000 0.0063
7.8312 32100 0.008
7.8556 32200 0.004
7.8800 32300 0.0057
7.9044 32400 0.005
7.9288 32500 0.0095
7.9532 32600 0.0042
7.9776 32700 0.0058
8.0020 32800 0.006
8.0263 32900 0.006
8.0507 33000 0.0054
8.0751 33100 0.0041
8.0995 33200 0.0045
8.1239 33300 0.0052
8.1483 33400 0.0067
8.1727 33500 0.008
8.1971 33600 0.0047
8.2215 33700 0.0079
8.2459 33800 0.0071
8.2703 33900 0.0043
8.2947 34000 0.0041
8.3191 34100 0.0035
8.3435 34200 0.0059
8.3679 34300 0.004
8.3923 34400 0.005
8.4167 34500 0.0067
8.4411 34600 0.0049
8.4655 34700 0.0034
8.4899 34800 0.0057
8.5143 34900 0.0052
8.5387 35000 0.005
8.5631 35100 0.0047
8.5875 35200 0.0089
8.6119 35300 0.0066
8.6363 35400 0.0044
8.6606 35500 0.0037
8.6850 35600 0.0059
8.7094 35700 0.0069
8.7338 35800 0.0069
8.7582 35900 0.0038
8.7826 36000 0.0028
8.8070 36100 0.0047
8.8314 36200 0.007
8.8558 36300 0.0036
8.8802 36400 0.0049
8.9046 36500 0.0041
8.9290 36600 0.0085
8.9534 36700 0.004
8.9778 36800 0.0044
9.0022 36900 0.0053
9.0266 37000 0.006
9.0510 37100 0.0051
9.0754 37200 0.0029
9.0998 37300 0.0041
9.1242 37400 0.0046
9.1486 37500 0.0057
9.1730 37600 0.0063
9.1974 37700 0.0048
9.2218 37800 0.0077
9.2462 37900 0.0056
9.2706 38000 0.0039
9.2949 38100 0.0036
9.3193 38200 0.0032
9.3437 38300 0.0055
9.3681 38400 0.0037
9.3925 38500 0.0045
9.4169 38600 0.0065
9.4413 38700 0.0047
9.4657 38800 0.0033
9.4901 38900 0.0052
9.5145 39000 0.0043
9.5389 39100 0.0043
9.5633 39200 0.0049
9.5877 39300 0.0074
9.6121 39400 0.0054
9.6365 39500 0.004
9.6609 39600 0.0031
9.6853 39700 0.0054
9.7097 39800 0.0061
9.7341 39900 0.0055
9.7585 40000 0.0033
9.7829 40100 0.0028
9.8073 40200 0.0046
9.8317 40300 0.0062
9.8561 40400 0.0033
9.8805 40500 0.0047
9.9049 40600 0.0045
9.9293 40700 0.0075
9.9536 40800 0.0035
9.9780 40900 0.0038

Framework Versions

  • Python: 3.10.12
  • Sentence Transformers: 3.3.1
  • Transformers: 4.47.0
  • PyTorch: 2.5.1+cu121
  • Accelerate: 1.2.1
  • Datasets: 4.0.0
  • Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
Downloads last month
2
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for codersan/eFuck

Base model

intfloat/multilingual-e5-small
Finetuned
(130)
this model

Papers for codersan/eFuck