Articles for people who
ship the stack
Rewrites on React, Node.js, TypeScript and AI. A translated body is shown when it exists. If it does not, you see the English text and a note that says so.
Tagged: Quantization
turbovec: Compress RAG Embeddings 16x With TurboQuant in Rust
turbovec brings Google's TurboQuant to Rust and shrinks RAG embedding indexes 16x with no training. Where it beats FAISS and when another tool fits better.
10 min readMiniCPM5-2B vs 4B Rivals: Why an Agentic Harness Tells a Different Story
MiniCPM5-2B beats 4B rivals on single-shot scores, then falls two to three times behind in agentic harnesses. The benchmark mix-up and Ollama throughput.
10 min readBitNet and Cassandra: How Ternary Weights Speed Up Local LLMs on CPUs
Six findings on 1.58-bit BitNet, speculative decoding and Cassandra: local LLM inference up to 6.17x faster on x86 CPUs, 82.2% less energy, same accuracy.
4 min readWhen a Model Never Finishes: Silent Failures Hidden From Hallucination Scores
Two checkpoints, two quantizations and two runtimes tested on invented names: one model stays silent for minutes, and a careful grader exposes what scores miss.
11 min readHow QLoRA Fine-Tunes Large Models on Small GPUs: 4-bit Basics
Understand QLoRA from the ground up: why full fine-tuning is expensive, how LoRA adapters and 4-bit quantization combine, and when this approach fits your project.
8 min readVector Quantization Compared: Int8 Versus Binary Recall and Storage
Measure how int8 and binary quantization change recall and storage for embeddings, why binary can collapse, and how to choose a safe compression level.
4 min readHardware reality check for running 100B-class models locally
Can a workstation run a 100B-class LLM locally? MoE sparsity, quantization, KV cache size and VRAM math decide it, not the parameter count in the headline.
11 min readWhen Quantization Passes Perplexity and Silently Breaks Model Safety
Low-bit KV cache quantization can wipe out a model's refusals while perplexity barely moves; here is why the standard metrics miss it and what to add to your gate.
7 min read
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.