Ollama

Ollama makes running open-source LLMs on your local machine as simple as a single terminal command. It handles model downloading, quantization, and GPU acceleration automatically. Supports Llama, Mistral, Gemma, Phi, and dozens of other models. Ollama exposes an OpenAI-compatible API, making it easy to swap between local and cloud AI in your applications. It's become the de facto standard for local AI development and privacy-focused deployments.