MLC LLM
Documentation: https://llm.mlc.ai/docs
Machine Learning Compilation for Large Language Models (MLC LLM) is a high-performance universal deployment solution that allows native deployment of any large language models with native APIs with compiler acceleration. The mission of this project is to enable everyone to develop, optimize and deploy AI models natively on everyone’s devices with ML compilation techniques.
Installation
MLC LLM is available via pip. It is always recommended to install it in an isolated conda virtual environment.
To verify the installation, activate your virtual environment, run
python -c "import mlc_llm; print(mlc_llm.__path__)"
You are expected to see the installation path of MLC LLM Python package.
Quick Start
Please check out our documentation for the quick start.
Introduction
Please check out our documentation for the introduction.
Links
- You might want to check out our online public Machine Learning Compilation course for a systematic walkthrough of our approaches.
- WebLLM is a companion project using MLC LLM’s WebGPU and WebAssembly backend.
- WebStableDiffusion is a companion project for diffusion models with the WebGPU backend.
Disclaimer
The pre-packaged demos are subject to the model License.