FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
No commits in 16 months — may not be actively maintained. See maintained alternatives →
Star Growth
Overview
FastChat 是一个开源平台，专门用于训练、服务和评估大型语言模型的聊天机器人。作为著名的 Chatbot Arena (lmarena.ai) 的技术支撑，FastChat 已为超过70个大型语言模型提供了超过1000万次聊天服务，并通过收集150万人类投票建立了权威的 LLM Elo 排行榜。该平台不仅包含了 Vicuna、MT-Bench 等先进模型的完整训练和评估代码，还提供了功能强大的分布式多模型服务系统，支持 Web UI 界面和 OpenAI 兼容的 RESTful API。FastChat 发布了多个重要资源，包括 LMSYS-Chat-1M 大规模对话数据集、Chatbot Arena 对话数据集等，为 LLM 研究社区提供了宝贵的数据和工具支持。
Deep Analysis
Powers Chatbot Arena (lmarena.ai) with 10M+ chat requests and 1.5M+ human votes — the de facto platform for LLM evaluation via crowdsourced human preference, plus an OpenAI-compatible serving layer for 70+ models
⚡ Capabilities
- • Training framework for LLM-based chatbots (Vicuna)
- • Distributed multi-model serving system with web UI
- • OpenAI-compatible RESTful API server
- • Chatbot Arena for LLM evaluation via human voting
- • MT-Bench multi-turn evaluation benchmark
- • Support for 70+ LLM models
- • Multi-GPU, CPU, Metal, XPU, and NPU inference
- • 8-bit quantization and CPU offloading
🔗 Integrations
✓ Best For
- ✓ Researchers evaluating and comparing LLM chatbot performance
- ✓ Teams needing OpenAI-compatible API serving for open-source models
- ✓ Running Chatbot Arena-style human evaluation campaigns
✗ Not Ideal For
- ✗ Production-grade LLM serving at scale (use vLLM or TGI directly)
- ✗ Teams only needing cloud API access to commercial models
Languages
Deployment
⚠ Known Limitations
- ⚠ Primarily research-focused; production serving may need additional engineering
- ⚠ Model weights (Vicuna) require Llama 2 license agreement
- ⚠ Web UI is basic compared to commercial chat interfaces
- ⚠ Arena leaderboard data may not reflect latest model capabilities
Pros
- + 业界权威的 LLM 评估平台，Chatbot Arena 排行榜是最受认可的模型性能参考标准
- + 完整的端到端解决方案，从模型训练、部署到评估全流程覆盖，支持 OpenAI 兼容 API
- + 活跃的开源生态和丰富的数据集资源，包括真实用户对话数据和人类偏好评估数据
Cons
- - 作为研究导向的平台，生产环境部署可能需要额外的稳定性和性能优化工作
- - 多模型服务系统的资源消耗较大，对硬件配置和运维能力有一定要求
Use Cases
- • LLM 研究者进行模型训练、微调和性能评估，特别是开发新的对话模型
- • 企业和开发者部署多模型聊天服务，提供统一的 API 接口支持多个 LLM
- • 教育和学术机构建立 LLM 评估基准，收集用户反馈数据进行模型对比分析
Getting Started
Alternatives
Works with FastChat
Tools that integrate with FastChat, often used together in the same stack.
Compare FastChat
Maintain FastChat?
Show your live rank in your README, or put FastChat in front of every visitor to AgentoolRank.