Open LLMs in Production — Serving, Quantization, Fine-Tuning and Cost
Open-weight language models have become genuinely good. Llama, Qwen, Mistral…
Tag
gpu
5 posts
Scaling on Kubernetes
A team moved their working single-GPU LLM service onto Kubernetes for…
The Real Cost in Rupees
A founder once told me, proudly, that self-hosting had cut their LLM costs by 8…
Memory Maths: Weights and KV Cache
A team once told me their 8B model “needs 80 GB of GPU memory” because it kept…
How LLM Inference Uses Your Hardware
When people first benchmark an LLM, they’re often confused by a result like this…