vllm?
#10
by
prudant
- opened
Hi, question:
ocker run
--rm
--gpus all
--network host
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server
it uses a custom image of vllm or the model is supported by the latest official image of vllm?
NICE&GREAT JOB !!!!!!
Thank you for your question. As the official vLLM doesn’t support this yet, we are currently using a custom image that we built in-house to include the necessary components.
thanks, great JOB!