Kinference introduces an open-source inference control plane for vLLM and KubernetesMachine translation
Kinference 发布开源推理控制平面，面向 vLLM 与 Kubernetes
Kinference is designed for teams building inference platforms with vLLM and Kubernetes. It provides token rate limiting, tenant usage and cost tracking, intelligent load balancing, and inference policies.
This source permits summary display only.
Read at the original source