Build my path

8a3e6f1lesson(tool-calling-contracts):free lesson · 25 min · works in your browser

Tool calls need contracts

A model asks to run your function by writing text that looks like JSON. You will write the contract that decides whether that request is allowed to run, and test it against seven real failure cases.

Part of Tool Calling That Holds Up. Sandboxes are deterministic simulations: no code runs on a server and no model is called.

Why this matters

The model writes the request. Your code decides if it runs.

When a model calls a tool it does not run anything. It writes a request: a tool name and some arguments, usually as JSON. Your code reads that request and runs the function. Everything that goes wrong in between is your responsibility, not the model's.

Models get arguments wrong in predictable ways: a required field is missing, a number arrives as a string, an enum value is invented, an amount is in dollars when you expected cents, or the JSON is cut off halfway. If your code runs whatever arrives, each of those becomes an incident.

A tool contract is a schema that your code checks before the tool runs. It is not a paragraph in the prompt asking the model to be careful. The prompt can describe the contract, but only code can enforce it.

Architecture · Where the contract sits
  1. ModelWrites a tool request as text
  2. ParserText to JSON, or a parse error
  3. ValidatorTypes, required fields, enums, ranges
  4. PolicyLimits and approval rules
  5. ApproverA person, only above the limit
  6. Toolissue_refund runs here, and only here

Blue edge: model · orange: tool · green: person · grey: data · plain: your code

Build step

Write the contract for issue_refund

Our demo support agent can refund an order. The contract says: order_id is required and looks like ord_ followed by four digits. amount_cents is a whole number from 1 to 50000. reason must be one of damaged, late, wrong_item or other. notify_customer is optional and defaults to false.

Amounts are in cents on purpose. Whole numbers avoid rounding errors, and the field name says the unit, so a model that writes 64.00 is caught by the integer check instead of refunding 64 cents or 6400 dollars.

The policy layer sits after validation. A valid refund above 20000 cents (200.00) is not rejected, it is held for a person to approve. Validation answers whether the request is well formed. Policy answers whether it is allowed right now.

When validation fails, send the errors back to the model as the tool result, in plain words it can act on, and let it try once more. If the second attempt also fails, stop and hand over to a person. Unlimited retries turn one bad call into a loop that burns tokens.

starter/handleToolCall.ts ts
export async function handleToolCall(raw: string) {  const call = JSON.parse(raw);            // throws on cut-off JSON  return issueRefund(call.arguments);     // runs whatever arrived}

The starter trusts the model. The solution parses, validates, applies policy, and returns errors the model can fix.

refundSchema.ts ts
export const refundSchema = {  order_id:        { type: "string",  required: true,  pattern: /^ord_[0-9]{4}$/ },  amount_cents:    { type: "integer", required: true,  min: 1, max: 50000 },  reason:          { type: "enum",    required: true,  values: ["damaged", "late", "wrong_item", "other"] },  notify_customer: { type: "boolean", required: false, default: false },} as const; export const APPROVAL_LIMIT_CENTS = 20000; // 200.00

Try it

Break the call and read what comes back

Edit the tool call below or load one of the recorded failures. The validator runs in your browser on every keystroke, exactly like the solution code above. Watch the message that would go back to the model: it should say what is wrong and how to fix it.

Notice that a prompt injection hidden in a free-text field does nothing here. The reason field only accepts four values, so a sentence asking for more refunds is rejected as an invalid reason instead of being obeyed.

sandbox/validate-tool-callruns in your browser · Illustrative
Rejected with message. Rejected with a message the model can act on.
  • amount_cents must be an integer number of cents, got "6400". Example: 6400 for 64.00.
to model → Error: amount_cents must be an integer number of cents, got "6400". Example: 6400 for 64.00. Fix the arguments and call again.

Test your result

Run the seven-case test set

Each check you switch on is a line of the solution code. The test set holds seven recorded tool calls with the result your system should produce: run, reject with a message, hold for approval, or hand over after a failed retry.

The starter state has only the JSON parse guard switched on and passes two of seven. Switch checks on until all seven pass, then compare with the solution. This is an evaluation harness in miniature: fixed inputs, expected outputs, and a score you can rerun after every change.

tests/tool-contract.specnot run yet
Checks in handleToolCall

What broke

What broke for other builders

  • Schema only in the prompt. The model was told amounts must be under 500. One day it was not. Limits written only in a prompt are suggestions.
  • Errors the model cannot use. Returning "invalid input" gives the model nothing to fix. Name the field, the rule and an example of a valid value.
  • Retry loops. Every failed call was sent back for another try, forever. Cap retries at one or two, then hand over to a person.
  • Unit confusion. A field called amount accepted 64.5. Was that dollars or cents? Put the unit in the field name and require whole numbers.
  • Free text where an enum belonged. A free-text reason field let an injected instruction ride along into logs and emails. Use an enum and keep free text out of actions.

Next commit

Next commit: retries that cannot double a refund

Your contract stops bad requests. It does not stop a good request from running twice when a network timeout makes your code retry. That needs idempotency keys and an approval boundary inside a traced agent run, which is the next lesson.

Quick check

1. Where should the rule "refunds over 200.00 need approval" live?
2. The model sends amount_cents: "6400" (a string). Best response?
3. Validation failed twice in a row. What now?

Ship the demo

  • ○ Test your result: every case passing at least once
  • ○ Quick check answered
Refund agent with limits: DraftSee it in Builds

Pass the tests and answer the quick check to unlock the merge.