Open Source: #serving

Catalog projects marked with #serving. Tags work as dedicated landing pages, so related tools are easier to find and connect.

This collection holds 2 projects with a combined 121,902 GitHub stars. Main languages: Python.

Filters

Reset
Languages

Repositories

Found: 2
vllm-project/vllm
vLLM

vLLM is a high-performance engine for LLM inference and serving with an OpenAI-compatible API, batching, and efficient memory management.

Stars 82,414 Forks 17,884 Author vllm-project Language Python License Apache-2.0
#ai #llm #inference #python #serving #machine-learning
Open
lm-sys/FastChat
FastChat

FastChat is a platform for training, serving, and evaluating large language models.

Stars 39,488 Forks 4,793 Author lm-sys Language Python License Apache-2.0
#llm #python #chatbot #serving
Open