Introduction to AI-driven systems philosophy and architecture
This book was written with the support of artificial intelligence tools for research, content structuring, and editing. The experience, methodologies, and strategic insights are the result of the author's practical work, while AI assisted in the writing process and organization of ideas.
To get the most out of this book, it's recommended to have familiarity with the following concepts and technologies:
In 2015, Google published a prophetic paper, "Hidden Technical Debt in Machine Learning," showing how in an ML application, the machine learning code was just a small black box at the center of a vast and complex infrastructure.
Ten years later, history repeats itself. The industry is in love with the promise of AI agents: a simple "magic box" where you insert an objective and extract value. But anyone who has tried to build a real application knows the truth. As Tomasz Tunguz writes, "What appeared to be a simple 'AI magic box' reveals itself to be an iceberg, with most of the engineering work hidden beneath the surface."
That submerged iceberg is made of context management, tool orchestration, memory systems, information retrieval (RAG), security guardrails, monitoring and, above all, managing the skyrocketing costs of APIs.
You won't find here another tutorial on how to make an API call. This is a strategic case study on how we built the hidden infrastructure, the 90% of work that allows the 10% of "AI magic" to function reliably and scalably.
We understood that to manage non-deterministic agents, which improvise and have "creative freedom," you don't need a better tool. You need a better organization, replicated in code. In these chapters, you'll discover how we built:
Director) that "hires" custom teams.Executor) that orchestrates the work.HolisticQualityAssuranceAgent) that evaluates business value.WorkspaceMemory) that allows the organization to learn.We built an "Agent Manager": an AI operating system that manages other agents, solving the problem of complexity and technical debt at the root. This manual is the story of how we succeeded, full of our scars and the lessons we learned. It's the guide for anyone who wants to stop playing with the tip of the iceberg and start building the submerged foundations.