← All TIL notes TIL

Strict JSON schemas cut function-calling errors more than prompt tweaks

Today I learned that tightening a tool’s JSON schema does more to reduce malformed function calls than any amount of prompt engineering.

Swapping a free-form status: string field for status: enum["pending", "done", "failed"], and marking previously-optional fields as required, cut malformed tool calls noticeably in our test set — with zero prompt changes.

Before:

{
  "name": "update_ticket",
  "parameters": {
    "type": "object",
    "properties": {
      "status": { "type": "string" }
    }
  }
}

After:

{
  "name": "update_ticket",
  "parameters": {
    "type": "object",
    "properties": {
      "status": { "type": "string", "enum": ["pending", "done", "failed"] }
    },
    "required": ["status"]
  }
}

Takeaway: before writing another paragraph of prompt instructions telling the model how to format a tool call, check whether the schema itself is loose enough to allow the malformed output in the first place.