#local-llm

Runs 70B models on a single 4 GB GPU without quantization, by keeping only one layer on the GPU at a time.

antirez's focused engine for running DeepSeek V4 Flash and a few other large open models locally on Metal, CUDA and ROCm.

One command to find which open-source LLMs actually run on your hardware, scored for fit, speed, quality and context.

Run, fine-tune and serve LLMs and diffusion models on your own hardware, with an OpenAI-compatible API for your agents.