Intro to AI Engineering

We are building an AI powered diagram design tool. You start with a fully functional Excalidraw canvas and a chat sidebar. Over the course of 12 lessons, you will turn that chat sidebar into an intelligent agent that can create, modify, and reason about diagrams on the canvas.

The app is built on Cloudflare Workers with the Agents SDK for the backend, and React with Excalidraw for the frontend. Everything runs locally on your machine. No deployment or cloud account needed.

Course Progression Roadmap 1. Lessons 1 through 5: Build the agent, wire up the chat, and establish evals so you have a working baseline to improve against. 2. Lessons 6 through 12: Systematically improve the agent using context engineering, better tools, RAG, generative UI, human-in-the-loop flows, planning mode, and data flywheels. Every improvement is measured with evals.

An AI Engineer is a system builder. You take foundation models (like Claude, GPT, etc.) and turn them into dependable product features. This is different from ML Engineering, where the focus is on training models, managing datasets, and optimizing model performance. As an AI Engineer, you treat the model as a given and focus on everything around it:

• Context engineering: curating exactly the right tokens to send to the model at inference time
• Tool design: giving the model the right capabilities to take actions in the world
• Evaluation: building automated test suites that measure whether the system is getting better or worse
• Production reliability: handling errors, managing costs, dealing with latency, and shipping to real users

*Think of it this way: an ML Engineer builds the engine. An AI Engineer builds the car around it.*

In 2026, roughly 70% of AI Engineer job postings center on four core skills: RAG, Evals, Agents, and Production Deployment.

The 12 Factor Agents framework provides a set of principles for building reliable AI agents. The most important takeaway: an agent is just a program. It has inputs, outputs, and a control flow that you design. The model is a powerful component inside that program, but you are the architect.

This course is structured around a simple iterative loop:

Build → Eval → Improve → Eval → Improve → ...

The first half of the course is about building and measuring. You will create the agent, wire up the chat interface, write a golden dataset of test cases, and build automated scorers. By the end of that phase you will have a working agent AND a baseline eval score.

The second half is about systematic improvement. Each lesson applies one technique, then runs evals to measure the impact. Every change is measured. This is the professional workflow: you never ship an improvement you have not evaluated.

Here is what the codebase looks like and what each piece does:

Follow these 5 steps to set up your environment: