Trong quá trình triển khai các mô hình AI trên nền tảng AWS, mình nhận thấy có một vấn đề lớn: các máy sử dụng GPU thường có chi phí khá cao nhưng hiệu quả mang lại chưa thật sự tương xứng, tình trạng bị lấy lại Spot Instance bất ngờ khiến việc duy trì […]

A/B testing has become a cornerstone of data-driven decision-making in the digital age. It’s a tool that empowers teams to evaluate ideas, combat biases, and uncover user insights. This article will explore the essence of A/B testing, from its foundational principles to practical applications, drawing lessons from industry expertise. What Is A/B Testing? At its […]

Part 2: System-Level and Deployment-Centric Optimization for LLMs In Part 1, we looked at the internal mechanics of big language models, concentrating on architectural strategies such as sparse attention, cache compression, and speculative decoding to reduce compute and memory costs during inference. However, even the most intelligent model optimizations can fail if the underlying system […]

Part 1: Architectural and Algorithmic Optimisations in LLM Inference What 20+ Papers and Open-Source Projects Taught Me About Cracking LLM Inference If you’re developing AI solutions and hosting foundation models built around large language models (LLMs), you should think about the cost of serving them. However, money is not the only consideration; believe me, if […]