FreeLevel 3 · Professional

AI Agents and Tool Use

An AI agent doesn't just answer; it acts, by calling tools in a loop. That makes it useful and dangerous. In this course you take apart two versions of Paystream's support agent, recorded step by step on 150 real requests that a support lead labelled. You'll design tools scoped to the logged-in customer, move business rules out of the prompt into tested code, and write a loop with limits, repeated-call detection and a trace. You'll find the refunds v1 issued without approval and the other customers' transfers it read, evaluate both versions on outcomes and on the paths they took, and see how instructions hidden in transfer narrations hijacked v1. Finally, you'll work out cost per correct resolution, design monitoring and a gradual rollout, and write the design for a safer v3. Every number comes from running the code on the recorded runs; live model calls are optional and shown with the Anthropic SDK.

About 7 hours
10 modules
1 badge
Final project
Optional verified certificate

Curriculum

01From Chatbot to AgentWhat an AI agent is (a model that chooses tools in a loop, while your code runs them), how it differs from a fixed workflow, and how to read the step-by-step record of an agent run.15 min
02Designing ToolsWrite the tools an agent can call, with clear names, descriptions and input schemas, scoped to the customer being served, and returning errors as data the model can act on.25 min
03The Agent LoopWrite the loop that runs an agent (send, check for tool calls, run them, send results back) with a step limit, a stop on repeated failures, and a full trace of every step.15 min
04Rules in Code, Not PromptsMove business rules out of the prompt and into deterministic tools the agent must call, check them against labelled decisions, and measure how often an agent that judged the rule itself got it wrong.25 min
05Permissions and ApprovalsClassify an agent's tools by the harm they can do, give it only the ones it needs, require a person's approval for risky actions, and measure the damage v1's broad permissions would have done.25 min
06Evaluating AgentsMeasure an agent on outcomes (did it take the right final action?) and on trajectories (did it get there safely?), break the results down by request type, and read the failures behind the numbers.25 min
07Injection Through ToolsSee how instructions hidden in data an agent reads (here, transfer narrations) can hijack it, measure how often v1 was hijacked, and build the layered defences that stopped v2.25 min
08Cost, Latency and LimitsWork out what an agent costs per request and per correct resolution, why each extra step costs more than the last, how long customers wait, and which limits keep both under control.15 min
09Monitoring Agents in ProductionDecide what to log, which numbers to watch every day, when to alert a person, and how to roll an agent out gradually, so that problems are found by your dashboard, not your customers.20 min
10Final ProjectPlan your final project, a support agent designed, tested on recorded runs, and made safe, with an evaluation and operating plan the head of support can sign off.20 min
Final assessmentPass it (60% or more) to earn the AI Agents and Tool Use course completion badge.

Loading…

AI Agents and Tool Use

About 7 hours

10 modules

1 badge

Final project

Optional verified certificate

Curriculum

01From Chatbot to AgentWhat an AI agent is (a model that chooses tools in a loop, while your code runs them), how it differs from a fixed workflow, and how to read the step-by-step record of an agent run.15 min

02Designing ToolsWrite the tools an agent can call, with clear names, descriptions and input schemas, scoped to the customer being served, and returning errors as data the model can act on.25 min

03The Agent LoopWrite the loop that runs an agent (send, check for tool calls, run them, send results back) with a step limit, a stop on repeated failures, and a full trace of every step.15 min

04Rules in Code, Not PromptsMove business rules out of the prompt and into deterministic tools the agent must call, check them against labelled decisions, and measure how often an agent that judged the rule itself got it wrong.25 min

05Permissions and ApprovalsClassify an agent's tools by the harm they can do, give it only the ones it needs, require a person's approval for risky actions, and measure the damage v1's broad permissions would have done.25 min

06Evaluating AgentsMeasure an agent on outcomes (did it take the right final action?) and on trajectories (did it get there safely?), break the results down by request type, and read the failures behind the numbers.25 min

07Injection Through ToolsSee how instructions hidden in data an agent reads (here, transfer narrations) can hijack it, measure how often v1 was hijacked, and build the layered defences that stopped v2.25 min

08Cost, Latency and LimitsWork out what an agent costs per request and per correct resolution, why each extra step costs more than the last, how long customers wait, and which limits keep both under control.15 min

09Monitoring Agents in ProductionDecide what to log, which numbers to watch every day, when to alert a person, and how to roll an agent out gradually, so that problems are found by your dashboard, not your customers.20 min

10Final ProjectPlan your final project, a support agent designed, tested on recorded runs, and made safe, with an evaluation and operating plan the head of support can sign off.20 min

Final assessmentPass it (60% or more) to earn the AI Agents and Tool Use course completion badge.