Run, fine-tune and serve LLMs and diffusion models on your own hardware, with an OpenAI-compatible API for your agents.
uv pip install unsloth --torch-backend=autoUnsloth is a local AI runtime and desktop app for running and training models. It serves them through an OpenAI-compatible API, so your agent tooling does not need to change.
unsloth start claude for this.uv pip install unsloth --torch-backend=auto
Or download the desktop app from the README.
AlexsJones · Tool / CLI
One command to find which open-source LLMs actually run on your hardware, scored for fit, speed, quality and context.
lyogavin · Tool / CLI
Runs 70B models on a single 4 GB GPU without quantization, by keeping only one layer on the GPU at a time.
antirez · Tool / CLI
antirez's focused engine for running DeepSeek V4 Flash and a few other large open models locally on Metal, CUDA and ROCm.