A Rust vector index with Python bindings that, the project says, fits 10M documents in about 4 GB and searches faster than FAISS.
pip install turbovecturbovec is a vector index built on Google Research's TurboQuant algorithm. It compresses embeddings heavily, so a 10M-document corpus that needs about 31 GB as float32 fits in about 4 GB.
pip install turbovec
Rust users can add it with cargo add turbovec.
Microsoft · Tool / CLI
Microsoft's lightweight Python tool that converts PDFs, Office files and more into clean Markdown for LLMs.
Firecrawl · Tool / CLI
The web context API that searches, scrapes and interacts with pages at scale, returning clean Markdown or structured data.
unclecode · Tool / CLI
An open-source, LLM-friendly web crawler and scraper that turns pages into clean Markdown and structured data.