AI Automation
Master Roadmap

Before writing a single prompt, understand what you’re prompting. LLMs don’t “know” things — they predict the next token given context. That mental model explains why they hallucinate, why temperature matters, why long prompts sometimes degrade quality. An hour understanding this saves you days of confused debugging later.

Prompt engineering is the highest-leverage skill in this entire roadmap. A well-structured system prompt can turn a mediocre model into a reliable automation. Learn to think of prompts as code: testable, versioned, and separable from the rest of your application. This week’s techniques will become second nature by Week 28.

Model selection is a cost and reliability decision, not just a performance one. Routing simple classification to Haiku while reserving Sonnet for complex reasoning can cut your API bill by 80%. This week builds the judgment to make that call — and to know when a local model on Ollama is actually the right answer.

Embeddings are the foundation of RAG — everything in Module 4 builds on what you learn here. Evals are what separate professional AI work from hobby projects. If you can’t measure whether a prompt works, you can’t improve it. Build the eval suite seriously this week; you’ll reuse the pattern for every module after this.

The first real API call feels like a milestone — and it should. This week is about getting completely comfortable with the SDK so the mechanics never slow you down again. Streaming in particular: a response that streams token-by-token feels alive compared to one that waits and dumps. Clients notice the difference immediately.

Tool use is the single most important API pattern in this roadmap. Once you understand the tool call loop — model requests a tool, you execute it, you return the result, model continues — you can build anything. Agents, pipelines, and autonomous workflows all boil down to this. Build it from scratch before using any framework.

Structured outputs are what make AI integration with real software reliable. Getting freeform text back from a model is fine for chat; getting a validated Pydantic object is what powers production systems. LangChain is worth knowing — not because you’ll always use it, but because understanding it helps you know when a simpler approach is better.

Hitting rate limits is a rite of passage — but it shouldn’t happen twice. Prompt caching alone can reduce your bill by 70% on pipelines with repeated system prompts. The batch API is the most underused feature in the whole ecosystem. This week is what separates engineers who can scale AI from ones who can only prototype.

Semantic search is a fundamentally different paradigm from keyword search. When you query “what causes inflation” and get back documents about price increases without the word “inflation” appearing, you understand why it matters. This week, embeddings go from a theoretical concept to something you’ve built and queried yourself.

Bad chunking is the most common reason RAG systems fail. If your chunks are too large, you retrieve irrelevant noise. Too small, and you lose context. This week you’ll run the experiments yourself — compare retrieval quality across chunk sizes — so the tradeoffs aren’t abstract. The metadata enrichment habits here pay off in Week 15.

Pure semantic search looks impressive in demos but fails on exact terms, product names, and dates. Hybrid search (semantic + BM25 + re-ranking) is the production standard for a reason — and the quality jump is dramatic. This week’s comparison project will show you the difference in numbers, which is exactly what you show a client.

This is a capstone week for the module — bring everything together into something portfolio-worthy. The RAGAS score gives you a number you can actually put in a proposal (“my RAG system scores 0.87 on faithfulness”). Citations make answers auditable. These aren’t polish — they’re what makes a system trustworthy enough to deploy with someone’s business data.

Build your first agent with no framework — just the SDK, a while loop, and a list of tools. It sounds simple, but writing it from scratch forces you to understand every decision: when does the loop end, how do tool results get fed back, what happens when a tool fails. That understanding is what lets you debug any framework confidently later.

An agent without memory is just an expensive chatbot. Context windows are your short-term memory — they’re also expensive and have hard limits. This week you’ll wire in a vector store for long-term memory and learn to summarise old context rather than truncate it. The personal assistant project is one of the most immediately useful things you’ll build.

Multi-agent is powerful but easy to over-engineer. The rule: only go multi-agent when a single agent genuinely can’t handle the task — specialisation, parallelism, or scale. The content pipeline project runs three agents in parallel and is actually faster than sequential. Feel that advantage concretely before designing systems with it.

Every serious agentic system needs guardrails. An agent that sends emails or modifies databases without approval checkpoints is a liability, not an asset. This week is about engineering reliability: confidence routing, sandboxed execution, tracing — the unglamorous work that makes agents safe enough to actually deploy. Don’t skip it.

n8n is worth investing in even if you’re primarily a Python developer. Clients who can’t review code can review an n8n workflow and actually understand what it does — that transparency builds trust. The AI nodes let you drop Claude into a pipeline without writing glue code. Build this week’s pipeline and you’ll immediately see where visual tools beat code, and where they don’t.

The trick most scraping tutorials miss: instead of writing fragile CSS selectors, pass the raw HTML to Claude and let it extract what you need. This pattern is remarkably resilient to layout changes. Playwright + AI extraction is one of the most durable web scraping approaches available — and surprisingly easy to sell as a monitoring or competitor intelligence service.

Most automation pipelines need to run reliably on a schedule — not once, but forever. The difference between a job that runs and one that runs reliably is idempotency, retries, and observability. Start with APScheduler before reaching for Celery. When you hit the wall APScheduler can’t handle, you’ll know exactly why Celery exists.

Every business has messy data. Using Claude to clean and standardise what rules can’t handle — mismatched date formats, inconsistent company names, unstructured notes — is one of the highest-value automations you can sell. The pandas + Claude pattern this week is commercially underrated and technically straightforward once you have it working.

FastAPI is the cleanest way to turn any AI capability into a callable service. The auto-generated /docs endpoint means you get a testable API interface for free. Learn streaming endpoints specifically — a chat endpoint that streams tokens over SSE is the baseline expectation for any AI product. This week’s project is the scaffolding you’ll use for Weeks 26–28.

Docker eliminates “works on my machine.” Once your app is containerised, it runs identically everywhere — your laptop, Railway, AWS, a client’s server. The goal this week is a live URL you can share. Not impressive local output — a real, accessible URL. That’s the delta between portfolio and just GitHub.

Most junior developers skip monitoring. Senior engineers treat it as table stakes. Being able to show a client a cost dashboard — “here’s exactly what each API call costs, and here’s how caching is reducing your bill” — is a differentiator that almost nobody else offers at your level. Langfuse tracing also cuts debugging time from hours to minutes.

This is the final week, and the capstone project is the most important deliverable in the entire roadmap. Take your best prior project and bring everything together: prompt versioning, full eval suite, Langfuse tracing, Docker, cost dashboard. What you end up with should be something you’re genuinely proud to show a potential client or employer — a complete, production-grade AI system.