Buoyant releases Kinference alpha for policy-based vLLM routing on KubernetesMachine translation

Buoyant has released Kinference under the Apache 2.0 license. The current version is v0.1.0-alpha.7, with policies that route requests based on tenant, requested model and GPU pool state. In Buoyant’s self-reported benchmark, median time to first token fell from 6.88 to 0.31 seconds on a shared-prefix workload; prompts without shared prefixes showed no significant median gain.