Update vLLM recipe

#4
by jeeejeee - opened
Files changed (2) hide show
  1. README.md +1 -25
  2. README_CN.md +1 -27
README.md CHANGED
@@ -167,31 +167,7 @@ For production serving, we recommend using [vLLM](https://github.com/vllm-projec
167
 
168
  ### vLLM
169
 
170
- Build vLLM from source:
171
- ```bash
172
- uv venv --python 3.12 --seed --managed-python
173
- source .venv/bin/activate
174
- git clone https://github.com/vllm-project/vllm.git
175
- cd vllm
176
- uv pip install --editable . --torch-backend=auto
177
- ```
178
-
179
- Start the vLLM server with MTP enabled:
180
-
181
- ```bash
182
- vllm serve tencent/Hy4-preview-FP8 \
183
- --tensor-parallel-size 8 \
184
- --speculative-config.method mtp \
185
- --speculative-config.num_speculative_tokens 3 \
186
- --attention-backend FLASHMLA_SPARSE \
187
- --tool-call-parser hy_v4 \
188
- --reasoning-parser hy_v4 \
189
- --enable-auto-tool-choice \
190
- --port 8000 \
191
- --served-model-name hy4-preview
192
- ```
193
-
194
- Or use official prebuilt image `vllm/vllm-openai:hy4-preview`:
195
 
196
  ```bash
197
  docker run --gpus all \
 
167
 
168
  ### vLLM
169
 
170
+ Use official prebuilt image `vllm/vllm-openai:hy4-preview`:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
171
 
172
  ```bash
173
  docker run --gpus all \
README_CN.md CHANGED
@@ -158,33 +158,7 @@ print(response.choices[0].message.content)
158
 
159
  ### vLLM
160
 
161
- 从源码构建 vLLM
162
-
163
- ```bash
164
- uv venv --python 3.12 --seed --managed-python
165
- source .venv/bin/activate
166
- git clone https://github.com/vllm-project/vllm.git
167
- cd vllm
168
- uv pip install --editable . --torch-backend=auto
169
- ```
170
-
171
- 启动 vLLM 服务,开启 MTP:
172
-
173
- ```bash
174
- vllm serve tencent/Hy4-preview-FP8 \
175
- --tensor-parallel-size 8 \
176
- --speculative-config.method mtp \
177
- --speculative-config.num_speculative_tokens 3 \
178
- --attention-backend FLASHMLA_SPARSE \
179
- --tool-call-parser hy_v4 \
180
- --reasoning-parser hy_v4 \
181
- --enable-auto-tool-choice \
182
- --port 8000 \
183
- --served-model-name hy4-preview
184
- ```
185
-
186
-
187
- 或者使用社区官方镜像部署:`vllm/vllm-openai:hy4-preview`:
188
 
189
  ```bash
190
  docker run --gpus all \
 
158
 
159
  ### vLLM
160
 
161
+ 使用社区官方镜像部署`vllm/vllm-openai:hy4-preview`:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
162
 
163
  ```bash
164
  docker run --gpus all \