aifollow.news
Back vLLM 发布记录
vLLM 发布记录· · Original publication time AI score54

vLLM releases v0.34.0 with extensible KV cache enabled by defaultMachine translation

vLLM 发布 v0.34.0,默认启用可扩展 KV 缓存

AI-assisted summary

vLLM v0.34.0 adds support for 10 models and sizes the KV cache using memory actually free after warmup. The change can improve throughput and reduce time to first token at high concurrency, but per-token decode latency can rise on some models. The older GPU runner has also been removed, so existing deployments need a compatibility check before upgrading.

The complete source text is not yet available.

Read at the original source
Found an error? Send a correction