The Technology Stack – Foundations

An architecture, no matter how brilliant, remains an abstract idea until it's built with concrete tools. The choice of these tools is never just a matter of technical preference; it's a statement of intent. Every technology we've chosen for this project was selected not only for its features, but for how it aligned with our philosophy of rapid, scalable, and AI-first development.

This chapter unveils the "building blocks" of our cathedral: the technology stack that made this architecture possible, and the strategic "why" behind every choice.

The Backend: FastAPI – The Mandatory Choice for Asynchronous AI

When building a system that must orchestrate dozens of calls to slow external services like LLMs, asynchronous programming isn't an option, it's a necessity. Choosing a synchronous framework (like Flask or Django in their classic configurations) would have meant creating an inherently slow and inefficient system, where every AI call would block the entire process.

FastAPI was the natural choice and, in our opinion, the only truly sensible one for an AI-driven backend.

The Frontend: Next.js – Separation of Concerns for Agility and UX

We could have served the frontend directly from FastAPI, but we made a deliberate strategic choice: completely separate the backend from the frontend.

Next.js (a React-based framework) allowed us to create an independent frontend application that communicates with the backend only through APIs.

The Database: Supabase – A "Backend-as-a-Service" for Speed

In an AI project, complexity is already sky-high. We wanted to minimize infrastructural complexity. Instead of managing our own PostgreSQL database, an authentication system, and a data API, we chose Supabase.

Supabase gave us the superpowers of a complete backend with the configuration effort of a simple database.

Vector Databases: The Brain Extension for AI Systems

Vector databases are a crucial component for the effectiveness of Large Language Model (LLM)-based systems, as they solve the limited context problem.

What it is and why it's useful

A vector database is a type of database specialized in storing, indexing, and searching embeddings. Embeddings are numerical representations (vectors) of objects, such as text, images, audio, or other data, that capture their semantic meaning. Two similar objects will have nearby vectors in space, while two very different objects will have distant vectors.

Their role is fundamental for allowing LLMs to access external information not contained in their training set. Instead of having to "remember" everything, the LLM can query the vector database to find the most relevant information based on the user's query. This process, called Retrieval-Augmented Generation (RAG), works like this:

1. The user's query is converted into an embedding (a vector).
2. The vector database searches for the most similar vectors (and therefore the semantically most relevant documents) to the query's vector.
3. The retrieved documents are provided to the LLM along with the original query, enriching its context and allowing it to generate a more precise and up-to-date response.

When to use one solution or another

In our case, we're using OpenAI's native vector database. This is a practical and fast choice, especially if you're already using the OpenAI SDK. It's useful for:

• Small-to-medium projects or proof-of-concepts.
• Simplifying architecture, avoiding the need to manage separate infrastructure.
• Native integration with the rest of the OpenAI ecosystem.

However, as we've rightly noted, you might want to consider dedicated solutions like Pinecone in the future. These options are often preferable for:

• Scalability and performance: they handle large volumes of data and high-speed queries.
• Control and flexibility: they offer more configuration, indexing, and data management options.
• Long-term costs: in some scenarios, self-hosted or dedicated solutions can be more cost-effective.

Coder CLI: Overcoming Context Limitations

Coder CLI (Command Line Interface) represents a significant evolution in LLM usage, transforming them from simple text generators to autonomous agents capable of action.

How it works and why it's effective

The main problem with LLMs is restricted context: they can only process a limited amount of text in a single input. Coder CLIs circumvent this limitation with an iterative and goal-based approach. Instead of receiving a single complex instruction, the CLI:

1. Receives a general objective (e.g., "Fix bug X").
2. Breaks down the objective into a series of smaller steps, creating a todo list.
3. Executes one command at a time in a controlled environment (e.g., a shell/bash).
4. Analyzes the output of each command to decide the next step.

This process of cascading reasoning allows the LLM to maintain focus, overcoming the limited context problem and tackling complex tasks that require multiple steps. The CLI can execute any shell/bash command, allowing it to:

• Read and write files (e.g., code, configurations).
• Interact with databases (executing Python scripts that read or write tables).
• Call external APIs to get or send data.
• Run automated tests to verify changes.

Potential and current limitations

Potential:

• Automatic fixing: the CLI can diagnose and fix bugs autonomously, running tests and iterating on the solution.
• Feature development: it can create scripts, modify application logic, and integrate it into existing code.
• Routine automation: it can handle repetitive tasks, like creating scripts for database management or log analysis.

Limitations and how we've worked around them:

• Holistic architectural approach: As we've observed, the LLM tends to focus on individual problems, without an overall vision. It often struggles to propose solutions that require extensive code or architectural reorganization.
• Targeted prompting (e.g., pillars): we've brilliantly worked around this limitation by providing specific and structured instructions. Using "pillars" or reasoning frameworks, we guide the LLM to consider broader aspects and not limit itself to the most immediate solution. This type of strategic prompting is essential for making the most of these tools' potential.

The CLI creates todo lists and executes them systematically, maintaining persistent context across multiple command executions. This allows complex multi-step operations that would be impossible with traditional LLM interactions. The ability to execute any shell command means the CLI can write Python scripts to interact with databases, make API requests, perform file operations, and even manage entire deployment processes.

From our experience, while architectural reasoning capabilities are still developing, the targeted prompting approach using structural frameworks (like our 15 pillars) significantly improves the quality of architectural decisions and helps maintain a holistic view of system design.

Development Tools: Claude CLI and Gemini CLI – Human-AI Co-Creation

Finally, it's essential to mention how this manual itself and much of the code were developed. We didn't use a traditional IDE in isolation. We adopted an approach of "pair programming" with command-line AI assistants.

This isn't just a technical detail, but a true development methodology that shaped the product.

This "AI-assisted" development approach allowed us to move at a speed unthinkable just a few years ago. We used AI not only as the object of our development, but as a partner in the creation process.

Chapter Conclusion

With this overview of our cathedral's "building blocks", the picture is complete. We've explored not only the abstract architecture, but also the concrete technologies and development methodologies that made it possible.

We're now ready for final reflections, to distill the most important lessons from this journey and look at what the future holds for us.