AI Efficiency ToolboxNewsletter
Back to Glossary
Glossary

Quantization

A plain-English explanation of quantized models.

AI Efficiency Toolbox logo

Why it matters

Quantization is why normal computers can run useful local AI.

Plain-English definition

A quantized model is a compressed model that usually uses less memory.

What to do next

Start with a common quantized model before trying the largest download.

Sources

Share