8GB to 70B: A Real Hardware Guide for Local LLMs
A real-world hardware guide for running local LLMs. I explain the effects of VRAM, quantization, CPU, and disk speed based on my own experiences. Budget and…
280 posts
Step-by-step guides, practical examples and hands-on tutorials.
A real-world hardware guide for running local LLMs. I explain the effects of VRAM, quantization, CPU, and disk speed based on my own experiences. Budget and…
In this guide, I'll walk you through setting up and running your own Large Language Model (LLM) on your local machine using Ollama. We'll do it in 5 simple.
Ensure your data privacy by setting up your own local LLM with Ollama and Open WebUI. A comprehensive guide.
I compare 3 common API versioning methods (URL Path, Query Parameter, Custom Header) for RESTful APIs. Which one is better in which situation...
Should prompt security strategies always be the same in AI applications? I share my flexible approaches and lessons learned for different scenarios.
I analyze the importance of switch hardening in network security and whether every device requires the same detailed configuration. Practical insights from my.
An in-depth analysis of AI agent tool-use architecture, its limitations, and costs. Featuring real-world scenarios and concrete data.
We examine the security of third-party dependencies used in our software projects and the associated costs for CI/CD processes with concrete examples.
Learn about idempotency in distributed systems, different approaches, and practical applications with Mustafa Erbay's experiences.
A guide to building a high-performance, low-cost search infrastructure using lightweight re-rankers, BM25, and PostgreSQL instead of expensive LLMs in RAG.
Are Zero Downtime Deployment (ZDD) strategies truly necessary for small and medium-sized projects? In this post, I'll discuss the costs and trade-offs from my.
I examined the impact of large language models (LLMs) on retrieval quality in Retrieval-Augmented Generation (RAG) systems. Real-world scenarios and concrete.
Why does Grafana's built-in alerting system fall short? A deep dive into Alertmanager installation, its advantages, and the ideal system architecture.
As a system architect for 20 years, I'm sharing the Linux commands that have saved me the most time, helped me solve the deepest problems, and are always at my.
Should monorepo build processes be managed with Makefiles or modern tools? A detailed comparison and experiences.
I examine the operational cost, trade-offs, and real-world impacts of detailed error handling. How much detail is necessary in which situations?
I examine sampling strategies in distributed tracing, balancing cost and detail loss based on my own experiences. Which approach works when?
Comparing JWT lifespans and secret rotation strategies, I'll share my experiences on which is more secure and practical in real-world scenarios.
I delve into the unending debate between SNMP and NetFlow in network monitoring, drawing from my own experiences. I discuss when I chose which, the trade-offs.
My personal experiences on choosing eventual consistency in distributed systems, the scalability advantages it brings, and the often overlooked operational.
Why point-to-point connections are insufficient in Enterprise Resource Planning (ERP) system integrations, illustrated with real-world examples and my.
API versioning is a challenge I frequently encounter in software architecture. In this post, I'll discuss different strategies, trade-offs, and my experiences.
Understanding the differences, advantages, disadvantages, and key considerations for making the right choice between eventual consistency and strong.
I examine the problems of unstructured logging I've encountered in systems, the parsing nightmare, and real-time analysis challenges through my own experiences.