File size: 564 Bytes
91533a3 57afcff 91533a3 57afcff |
1 2 3 4 5 6 7 8 9 |
# DeepSeek-V3-NextN
NextN layer exported for [deepseek-ai/DeepSeek-V3](https://huggingface.co/deepseek-ai/DeepSeek-V3) model with [script](https://github.com/sgl-project/sglang/blob/main/scripts/export_deepseek_nextn.py). Used for speculative decoding in [SGLang](https://github.com/sgl-project/sglang).
## Usage
```
python3 -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --speculative-algo EAGLE --speculative-draft lmsys/DeepSeek-V3-NextN --speculative-num-steps 2 --speculative-eagle-topk 4 --speculative-num-draft-tokens 4 --trust-remote --tp 8
```
|