MLC Models

Available Models

Model ID	Quantization	Link
Llama-3-8B-Instruct	q0f16	HuggingFace
Llama-3-8B-Instruct	q3f16_1	HuggingFace
Llama-3-8B-Instruct	q4f16_1	HuggingFace
Llama-3-8B-Instruct	q4f32_1	HuggingFace
Hermes-2-Pro-Llama-3-8B	q0f16	HuggingFace
Hermes-2-Pro-Llama-3-8B	q3f16_1	HuggingFace
Hermes-2-Pro-Llama-3-8B	q4f16_1	HuggingFace
Hermes-2-Pro-Llama-3-8B	q4f32_1	HuggingFace
Hermes-2-Theta-Llama-3-70B	q0f16	HuggingFace
Hermes-2-Theta-Llama-3-70B	q3f16_1	HuggingFace
Hermes-2-Theta-Llama-3-70B	q4f16_1	HuggingFace
Hermes-2-Theta-Llama-3-8B	q0f16	HuggingFace
Hermes-2-Theta-Llama-3-8B	q3f16_1	HuggingFace
Hermes-2-Theta-Llama-3-8B	q4f16_1	HuggingFace
Hermes-2-Theta-Llama-3-8B	q4f32_1	HuggingFace
Phi-3-mini-128k-instruct	q0f16	HuggingFace
Phi-3-mini-128k-instruct	q4f16_1	HuggingFace
Phi-3-mini-128k-instruct	q4f32_1	HuggingFace
Mistral-7B-Instruct-v0.3	q0f16	HuggingFace
Mistral-7B-Instruct-v0.3	q3f16_1	HuggingFace
Mistral-7B-Instruct-v0.3	q4f16_1	HuggingFace
Mistral-7B-Instruct-v0.3	q4f32_1	HuggingFace
Qwen1.5-0.5B-Chat	q0f16	HuggingFace
Qwen1.5-0.5B-Chat	q4f16_1	HuggingFace
Qwen1.5-0.5B-Chat	q4f32_1	HuggingFace
Qwen1.5-1.8B-Chat	q0f16	HuggingFace
Qwen1.5-1.8B-Chat	q4f16_1	HuggingFace
Qwen1.5-1.8B-Chat	q4f32_1	HuggingFace
Qwen2-0.5B-Instruct	q0f16	HuggingFace
Qwen2-0.5B-Instruct	q0f32	HuggingFace
Qwen2-0.5B-Instruct	q4f16_1	HuggingFace
Qwen2-0.5B-Instruct	q4f32_1	HuggingFace
Qwen2-1.5B-Instruct	q0f16	HuggingFace
Qwen2-1.5B-Instruct	q4f16_1	HuggingFace
Qwen2-1.5B-Instruct	q4f32_1	HuggingFace
Qwen2-72B-Instruct	q0f16	HuggingFace
Qwen2-72B-Instruct	q4f16_1	HuggingFace
Qwen2-72B-Instruct	q4f32_1	HuggingFace
Qwen2-7B-Instruct	q0f16	HuggingFace
Qwen2-7B-Instruct	q4f16_1	HuggingFace
Qwen2-7B-Instruct	q4f32_1	HuggingFace
Mixtral-8x7B-Instruct-v0.1	q0f16	HuggingFace
Mixtral-8x7B-Instruct-v0.1	q4f16_1	HuggingFace
Mixtral-8x7B-Instruct-v0.1	q4f32_1	HuggingFace