Spaces:
Running
Running
Feature(LLMLingua): update readme
Browse files
README.md
CHANGED
|
@@ -11,3 +11,29 @@ license: mit
|
|
| 11 |
---
|
| 12 |
|
| 13 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
<div style="display: flex; align-items: center; ">
|
| 17 |
+
<div style="width: 100px; margin-right: 10px; height:auto;" align="left">
|
| 18 |
+
<img src="images/LLMLingua_logo.png" alt="LLMLingua" width="100" align="left">
|
| 19 |
+
</div>
|
| 20 |
+
<div style="flex-grow: 1;" align="center">
|
| 21 |
+
<h2 align="center">LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models & LongLLMLingua</h1>
|
| 22 |
+
</div>
|
| 23 |
+
</div>
|
| 24 |
+
|
| 25 |
+
<p align="center">
|
| 26 |
+
| <a href="https://arxiv.org/abs/2310.05736"><b>LLMLingua Paper</b></a> | <a href="https://arxiv.org/abs/2310.06839"><b>LongLLMLingua Paper</b></a> | <a href="https://huggingface.co/spaces/microsoft/LLMLingua"><b>HF Space Demo</b></a> |
|
| 27 |
+
</p>
|
| 28 |
+
|
| 29 |
+
## Tl;DR
|
| 30 |
+
|
| 31 |
+
LLMLingua, that uses a well-trained small language model after alignment, such as GPT2-small or LLaMA-7B, to detect the unimportant tokens in the prompt and enable inference with the compressed prompt in black-box LLMs, achieving up to 20x compression with minimal performance loss.
|
| 32 |
+
|
| 33 |
+
[LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models](https://arxiv.org/abs/2310.05736) (EMNLP 2023).<br>
|
| 34 |
+
_Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang and Lili Qiu_
|
| 35 |
+
|
| 36 |
+
LongLLMLingua is a method that enhances LLMs' ability to perceive key information in long-context scenarios using prompt compression, achieveing up to $28.5 in cost savings per 1,000 samples while also improving performance.
|
| 37 |
+
|
| 38 |
+
[LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression](https://arxiv.org/abs/2310.06839) (Under Review).<br>
|
| 39 |
+
_Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang and Lili Qiu_
|