CAT 
A deep dive into model quantization with GGUF and llama.cpp and model evaluation with LlamaIndex
Originally appeared here:
Democratizing LLMs: 4-bit Quantization for Optimal LLM Inference
Go Here to Read this Fast! Democratizing LLMs: 4-bit Quantization for Optimal LLM Inference
