Spaces:
Runtime error
Runtime error
Feature(MInference): style
Browse files
app.py
CHANGED
|
@@ -14,9 +14,7 @@ HF_TOKEN = os.environ.get("HF_TOKEN", None)
|
|
| 14 |
|
| 15 |
|
| 16 |
DESCRIPTION = """
|
| 17 |
-
|
| 18 |
-
<h1>MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention (Under Review, ES-FoMo @ ICML'24)</h1>
|
| 19 |
-
</div>
|
| 20 |
|
| 21 |
_Huiqiang Jiang†, Yucheng Li†, Chengruidong Zhang†, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang and Lili Qiu_
|
| 22 |
|
|
@@ -24,10 +22,8 @@ _Huiqiang Jiang†, Yucheng Li†, Chengruidong Zhang†, Qianhui Wu, Xufang Luo
|
|
| 24 |
<a href="https://aka.ms/MInference" target="blank"> [Project Page]</a>
|
| 25 |
<a href="https://arxiv.org/abs/2407" target="blank"> [Paper]</a></h3>
|
| 26 |
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
<li><p><b>🧩</b>  We will present <font color="#337ab7"><b>MInference 1.0</b></font> at the <font color="#337ab7"><b>Microsoft Booth</b></font> and <font color="#337ab7"><b>ES-FoMo</b></font> at <b>ICML'24</b>. See you in Vienna!.
|
| 30 |
-
</ol>
|
| 31 |
|
| 32 |
<font color="brown"><b>This is only a deployment demo. Due to limited GPU resources, we do not provide an online demo. You will need to follow the code below to try MInference locally.</b></font>
|
| 33 |
|
|
@@ -38,7 +34,6 @@ pip install -r requirments.txt
|
|
| 38 |
pip install flash_attn pycuda==2023.1
|
| 39 |
python app.py
|
| 40 |
```
|
| 41 |
-
<br/>
|
| 42 |
"""
|
| 43 |
|
| 44 |
LICENSE = """
|
|
|
|
| 14 |
|
| 15 |
|
| 16 |
DESCRIPTION = """
|
| 17 |
+
# MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention (Under Review, ES-FoMo @ ICML'24)
|
|
|
|
|
|
|
| 18 |
|
| 19 |
_Huiqiang Jiang†, Yucheng Li†, Chengruidong Zhang†, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang and Lili Qiu_
|
| 20 |
|
|
|
|
| 22 |
<a href="https://aka.ms/MInference" target="blank"> [Project Page]</a>
|
| 23 |
<a href="https://arxiv.org/abs/2407" target="blank"> [Paper]</a></h3>
|
| 24 |
|
| 25 |
+
## News
|
| 26 |
+
- 🧩 We will present **MInference 1.0** at the _**Microsoft Booth**_ and _**ES-FoMo**_ at ICML'24. See you in Vienna!
|
|
|
|
|
|
|
| 27 |
|
| 28 |
<font color="brown"><b>This is only a deployment demo. Due to limited GPU resources, we do not provide an online demo. You will need to follow the code below to try MInference locally.</b></font>
|
| 29 |
|
|
|
|
| 34 |
pip install flash_attn pycuda==2023.1
|
| 35 |
python app.py
|
| 36 |
```
|
|
|
|
| 37 |
"""
|
| 38 |
|
| 39 |
LICENSE = """
|