Small-model tool-calling: the SOTA playbook (what others do)

What: a survey of how the field builds SOTA small (1-8B) tool-callers — data, SFT tricks, RL, eval, on-device serving — distilled to what’s stealable on a Mac. Why: we kept finding cheap wins one at a time (parser fixes, function masking) because we worked bottom-up. This is the top-down map so we stop re-deriving the known playbook. Surveyed 2026-06-14.

Companion: tool-calling-frontier-parity.md (our own results) · external-references.md.

1. Data synthesis (how SOTA training sets are built)

• APIGen (xLAM-60k): 3,673 executable APIs, three-stage verification — format → actual execution → LLM-semantic. The execution gate is the key idea. paper
• APIGen-MT: multi-turn via “blueprint (committee of LLM reviewers + ground-truth actions) → simulated human-agent trajectory.” Beats GPT-4o/Claude-3.5 on τ-bench. paper
• ToolACE (the set we used): self-evolving synthesis → 26k diverse APIs, multi-agent user/assistant/verifier, complexity-tiered, dual-layer (rule + model) verification. paper
• ToolBench/ToolLLM: 16k APIs, DFSDT search for valid multi-step trajectories. paper

2. SFT tricks

• Function masking (Hammer): randomize ~33-50% of function/param names during training → model relies on descriptions, not memorized names. Robust to unseen catalogs. paper
• Irrelevance/abstention augmentation (Hammer): ~10% of data = correct tool removed, label = empty list → teaches “don’t call when nothing fits.” Cuts over-emission.
• Decontamination + curriculum (single→multi→long-horizon). LoRA suffices ≥8B; full-FT common at 1.7-4B. SOTA small bases: Qwen2.5 / Llama-3.1-8B.

3. RL / post-training

• Graded reward >> binary (ToolRL): reward in [-3,3] = tool-name (Jaccard) + param-name (Jaccard) + param-value (exact) + a separate format reward. Fine-grained partial credit beats binary at all sizes; length rewards hurt small models. ToolRL ≈ +10 over SFT. paper
• Single-turn RL barely moves (EGPO got +2.15 on single-turn BFCL — matches our +2). Gains live in reward granularity + multi-call credit + cold-start. EGPO
• Stability/efficiency (DAPO/Dr.GRPO): clip-higher (ε_hi≈0.28), drop std-normalization, token-level loss, and dynamic sampling — discard prompts where all K rollouts pass or all fail (we saw many such no-variance skips). Small batch is the #1 mistake. GRPO tricks
• RFT (rejection-sampling best-of-N SFT, filtered by the AST matcher) is the cheap middle rung between SFT and full GRPO. Cold-start GRPO can beat SFT-init GRPO (SFT overfits). DPO/KTO cheaper but generalize worse on multi-step. post-training 2026
• Credit assignment for multi-call (CARL): segment-level advantages so a good and bad call in one trajectory get opposite credit — but the full version is cluster-scale. CARL

4. Eval (and its traps)

• Benchmarks: BFCL v3/v4 (AST + executable + irrelevance/relevance; v4 adds agentic + format-sensitivity), τ-bench/τ²-bench (multi-turn, stateful, Pass^k reliability), HammerBench, ACEBench, MCP-Bench (real MCP schemas). BFCL · τ²
• Traps: gold under-determination + exact-match brittleness (our hermes lesson); contamination (FC specialists train on BFCL-like data → high scores partly by-design); prompt-format sensitivity drops AST 13-19 pts under paraphrase.
• Our frontier-calibration gate (“~100% or broken”) is industry-aligned.
• Small models cliff on multi-turn: Command-R7B 69%→5% single→multi-turn; Llama-3.1-8B 61%→9.6%. Purpose-tuned (xLAM-2-8b-fc) survive ~69%. Our single-turn 78 likely overstates real agentic ability — we haven’t tested multi-turn at all.

5. On-device / Apple-Silicon serving

• Quantization hurts FC disproportionately — tool selection + arg fidelity degrade faster than general tasks at 4-bit. Prefer 8-bit / AWQ / QAT for the tool-caller, BFCL-validate. study
• Constrained decoding: guarantees valid JSON; rescues weak small models (3B beat 70B on BFCL under XGrammar), but a “constraint tax” — hard schema-locking can suppress reasoning (Qwen2.5-1.5B 91.5%→48%). Adopt with a free-text reasoning prefix, not a hard global lock. XGrammar · constraint tax
• Prefix/KV-cache the repeated tool-schema system prompt — biggest speed win locally.
• Rapid-MLX (17 tool-parsers w/ auto-recovery, batching, ~2× Ollama) / oMLX / LM Studio. Rapid-MLX

STEAL / ADOPT priority list (mapped to our gaps)

1. ToolRL graded reward (name+param-name+param-value+format) replacing our per-call binary — the single highest-ROI fix to our flat +2 RL. Reward-fn edit only.
2. Function masking + irrelevance augmentation in SFT data — targets our live_multiple WRONG_FUNC (32%) + over-emission directly. Cheap data edit.
3. Dynamic sampling (drop zero-advantage groups) + clip-higher — fixes the no-variance skips we observed; sample-efficiency on Mac compute. Few-line GRPO edit.
4. 8-bit (not 4-bit) for the tool-caller, BFCL-validated — 4-bit may be silently costing FC accuracy.
5. Eval upgrades: add irrelevance + format-sensitivity (paraphrase ×3) probes now; a multi-turn stateful slice later (where small models actually cliff).
6. RFT (best-of-N filtered by the AST matcher) as a cheap rung before full GRPO.
7. Serving: prefix-KV-cache the tool schema; constrained JSON with a reasoning prefix.