Update README.md
Browse files
README.md
CHANGED
|
@@ -16,6 +16,23 @@ tags:
|
|
| 16 |
- neural-compressor
|
| 17 |
---
|
| 18 |
# INT8 GPT-J 6B
|
|
|
|
|
|
|
| 19 |
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
|
| 20 |
|
| 21 |
-
This int8 PyTorch model is generated by [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
- neural-compressor
|
| 17 |
---
|
| 18 |
# INT8 GPT-J 6B
|
| 19 |
+
|
| 20 |
+
## Model Description
|
| 21 |
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
|
| 22 |
|
| 23 |
+
This int8 PyTorch model is generated by [intel-extension-for-transformers](https://github.com/intel/intel-extension-for-transformers).
|
| 24 |
+
| Package | Version |
|
| 25 |
+
|----------------------|------------|
|
| 26 |
+
| intel-extension-for-transformers| a4aba8ddb07c9b744b6ac106502ec059e0c47960 |
|
| 27 |
+
| neural-compressor | 2.4.1 |
|
| 28 |
+
| torch | 2.1.0+cpu |
|
| 29 |
+
| intel-extension-for-pytorch | 2.1.0 |
|
| 30 |
+
| transformers | 4.32.0 |
|
| 31 |
+
|
| 32 |
+
## Evaluation results
|
| 33 |
+
Evaluating the accuracy of the optimized model of gpt-j-6b using the lambada_openai dataset in lm_eval.
|
| 34 |
+
|
| 35 |
+
| Dtype | Dataset | Precision |
|
| 36 |
+
|------ |--------|--------|
|
| 37 |
+
| FP32 |Lambada_openai | 0.6831 |
|
| 38 |
+
| INT8 |Lambada_openai | 0.6835 |
|