Hugging Face's logo

RedHatAI
/
bge-base-en-v1.5-quant

Feature Extraction
Transformers
ONNX
English
bert
mteb
sparse
sparsity
quantized
embeddings
int8
deepsparse
Eval Results
text-embeddings-inference
Model card Files
xet
Community
1
zeroshot commited on
Commit
1da2136
·
1 Parent(s): 3a18385

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +167 -0
README.md CHANGED
@@ -4,6 +4,173 @@ language:
4
  - en
5
  tags:
6
  - sparse sparsity quantized onnx embeddings int8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  ---
8
  This is the quantized (INT8) ONNX variant of the [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) embeddings model created with [DeepSparse Optimum](https://github.com/neuralmagic/optimum-deepsparse) for ONNX export/inference pipeline and Neural Magic's [Sparsify](https://github.com/neuralmagic/sparsify) for one-shot quantization.
9
 
 
4
  - en
5
  tags:
6
  - sparse sparsity quantized onnx embeddings int8
7
+ model-index:
8
+ - name: bge-base-en-v1.5-quant
9
+ results:
10
+ - task:
11
+ type: Classification
12
+ dataset:
13
+ type: mteb/amazon_counterfactual
14
+ name: MTEB AmazonCounterfactualClassification (en)
15
+ config: en
16
+ split: test
17
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
18
+ metrics:
19
+ - type: accuracy
20
+ value: 76.16417910447761
21
+ - type: ap
22
+ value: 39.62965026785565
23
+ - type: f1
24
+ value: 70.30041589476463
25
+ - task:
26
+ type: Classification
27
+ dataset:
28
+ type: mteb/amazon_polarity
29
+ name: MTEB AmazonPolarityClassification
30
+ config: default
31
+ split: test
32
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
33
+ metrics:
34
+ - type: accuracy
35
+ value: 92.95087500000001
36
+ - type: ap
37
+ value: 89.92451248271642
38
+ - type: f1
39
+ value: 92.94162732408543
40
+ - task:
41
+ type: Classification
42
+ dataset:
43
+ type: mteb/amazon_reviews_multi
44
+ name: MTEB AmazonReviewsClassification (en)
45
+ config: en
46
+ split: test
47
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
48
+ metrics:
49
+ - type: accuracy
50
+ value: 48.214
51
+ - type: f1
52
+ value: 47.57084372829096
53
+ - task:
54
+ type: Classification
55
+ dataset:
56
+ type: mteb/banking77
57
+ name: MTEB Banking77Classification
58
+ config: default
59
+ split: test
60
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
61
+ metrics:
62
+ - type: accuracy
63
+ value: 86.35064935064935
64
+ - type: f1
65
+ value: 86.32782396028989
66
+ - task:
67
+ type: Classification
68
+ dataset:
69
+ type: mteb/emotion
70
+ name: MTEB EmotionClassification
71
+ config: default
72
+ split: test
73
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
74
+ metrics:
75
+ - type: accuracy
76
+ value: 51.269999999999996
77
+ - type: f1
78
+ value: 45.9714399031315
79
+ - task:
80
+ type: Classification
81
+ dataset:
82
+ type: mteb/imdb
83
+ name: MTEB ImdbClassification
84
+ config: default
85
+ split: test
86
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
87
+ metrics:
88
+ - type: accuracy
89
+ value: 89.7204
90
+ - type: ap
91
+ value: 85.70238397381907
92
+ - type: f1
93
+ value: 89.70961232185473
94
+ - task:
95
+ type: Classification
96
+ dataset:
97
+ type: mteb/mtop_domain
98
+ name: MTEB MTOPDomainClassification (en)
99
+ config: en
100
+ split: test
101
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
102
+ metrics:
103
+ - type: accuracy
104
+ value: 93.95120839033288
105
+ - type: f1
106
+ value: 93.70348712248138
107
+ - task:
108
+ type: Classification
109
+ dataset:
110
+ type: mteb/mtop_intent
111
+ name: MTEB MTOPIntentClassification (en)
112
+ config: en
113
+ split: test
114
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
115
+ metrics:
116
+ - type: accuracy
117
+ value: 75.25763793889648
118
+ - type: f1
119
+ value: 57.59583082574482
120
+ - task:
121
+ type: Classification
122
+ dataset:
123
+ type: mteb/amazon_massive_intent
124
+ name: MTEB MassiveIntentClassification (en)
125
+ config: en
126
+ split: test
127
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
128
+ metrics:
129
+ - type: accuracy
130
+ value: 75.16476126429052
131
+ - type: f1
132
+ value: 73.29287381030854
133
+ - task:
134
+ type: Classification
135
+ dataset:
136
+ type: mteb/amazon_massive_scenario
137
+ name: MTEB MassiveScenarioClassification (en)
138
+ config: en
139
+ split: test
140
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
141
+ metrics:
142
+ - type: accuracy
143
+ value: 78.9340954942838
144
+ - type: f1
145
+ value: 79.04036413238218
146
+ - task:
147
+ type: Classification
148
+ dataset:
149
+ type: mteb/toxic_conversations_50k
150
+ name: MTEB ToxicConversationsClassification
151
+ config: default
152
+ split: test
153
+ revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
154
+ metrics:
155
+ - type: accuracy
156
+ value: 72.123
157
+ - type: ap
158
+ value: 14.396060207954983
159
+ - type: f1
160
+ value: 55.24344377812756
161
+ - task:
162
+ type: Classification
163
+ dataset:
164
+ type: mteb/tweet_sentiment_extraction
165
+ name: MTEB TweetSentimentExtractionClassification
166
+ config: default
167
+ split: test
168
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
169
+ metrics:
170
+ - type: accuracy
171
+ value: 59.67176004527447
172
+ - type: f1
173
+ value: 59.97320225890037
174
  ---
175
  This is the quantized (INT8) ONNX variant of the [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) embeddings model created with [DeepSparse Optimum](https://github.com/neuralmagic/optimum-deepsparse) for ONNX export/inference pipeline and Neural Magic's [Sparsify](https://github.com/neuralmagic/sparsify) for one-shot quantization.
176