Mac-local AI mastery map

The north-star (AGENTS.md): be best-in-class at Mac-local AI, learn the whole space like a sponge (including the single-machine ↔ distributed boundary), build everything buildable on a Mac, and position for future scale. This is the living agenda — the checklist of what to learn/build and what’s already covered.

Legend: ✅ built/learned · 🟡 partial / scaffolded · ⬜ to learn or build. Each area links the canonical doc/source — we don’t re-teach here (DRY).

1. Foundations

The ground-up curriculum — mostly ✅. See curriculum and sessions 01–08

• llm-mechanics-fundamentals.

• ✅ NN basics, gradient descent, non-linearities, ML paradigms, scaling, tokenization/embeddings, behavior learning, training mechanics.
• ✅ Modern architecture (RoPE, RMSNorm, SwiGLU, GQA, MoE, attention-as-matmuls) — see advanced-ml-systems-eval §1–4.

2. Training & post-training

Canonical: advanced-llm-training.

• ✅ From-scratch pretrain (tiny), SFT, DPO, finetune — tinygpt train/sft/dpo/finetune.
• ✅ PEFT zoo: LoRA, DoRA, QLoRA, VeRA, PISSA, LoftQ, AdaLoRA — SFT.swift.
• ✅ Distillation / cost-compression — validated: a 0.6B matched a 4B on tool-calling at 1/7th size. tinygpt distill. This is the live winning lane.
• ✅ Synthetic data (magpie), quality classifier + filter.
• 🟡 RL: GRPO-on-clarify (docs/GRPO_CLARIFY.md); reward modeling (B28); multi-turn agentic distillation → frontier-parity (4B 58→100); teacher-free ReST ran (iter-1: breadth 60→65%, depth held, recovered the 31% negative-transfer collapse) (self-improving-agents, journey §8).
• ⬜ From VibeThinker (Spectrum-to-Signal): diversity-exploring distillation + MGPO auto-curriculum + specialist weight-merging (the direct antidote to the measured negative transfer) — diversity doc.
• ⬜ Learn deeper: full GRPO/PPO loop, reward over-optimization, WSD schedule (B11), layer-wise LR decay (B15), micro-automixer (B21).

3. Inference & serving

Canonical: advanced-llm-inference.

• ✅ Sampling, speculative decoding (B14), KV-quant, ANE/CoreML serve, GGUF load.
• ✅ Browser WebGPU inference (hand-written WGSL kernels) + transformers.js page.
• 🟡 tinygpt serve HF path is unoptimized (~7 tok/s on 4B) — known gap.
• ⬜ Learn/build (steal from oMLX): continuous batching (B34, for eval), tiered KV cache RAM→SSD, persistent prefix-KV cache, PagedAttention, FlashAttention internals.

4. Evaluation & judgment ← the differentiated moat

Canonical: advanced-ml-systems-eval §11–13.

• ✅ Harnesses: BFCL, τ-bench, HumanEval, lm-eval, MTEB, Indic; eval-compare.
• ✅ eval-gate (B32, CI regression gate) + mac-assistant-judgment benchmark (novel, unpublished-elsewhere baselines).
• 🟡 LLM-as-judge (E7); perplexity/contamination (theory in the eval doc).
• ⬜ Learn/build: judge-bias mitigation, RL environments as reward functions (the Prime Intellect angle), agent-eval-protocol (B23).

5. Interpretability

• ✅ SAE + SAE-explore, ROME, MEMIT, tuned-lens, linear-probe, causal-trace, patch.
• 🟡 SAE→SAELens/Neuronpedia export (B17); interp-on-checkpoints timeline (B13).

6. Modalities

• ✅ Text; embeddings + rerankers (rerank-train/eval, MTEB).
• 🟡 VLM (qwen3-vl-mrope-deepstack; vlm-smoke).
• 🟡 Speech (Pace): Apple Speech STT + Kokoro TTS shipping; WhisperKit streaming ⬜.
• See speech-and-systems-topics for the voice stack.

7. Agents & tools

Canonical: advanced-ml-systems-eval §9–10, agent-context-hierarchy, model-vs-agent.

• ✅ Tool-calling, tinygpt agent, deferred tools (B26 🟡), Pace’s plan-act-observe loop.
• 🟡 Mini-router family (B2–B7).
• ⬜ Trajectory recorder (B22), agent-eval protocol (B23), RAG vector layer.

8. Compression & efficiency ← the validated lane

• ✅ Quantization (HQQ, GPTQ, 4-bit), structured/unstructured pruning.
• ✅ Distillation (big→small for cost) — proven on this Mac.
• ⬜ Learn/build: logit-level (soft) distillation, fp8, KV-quant tradeoffs, energy-per-token (B9).

9. Distribution & packaging

• ✅ Model gallery + project pins (B31), quickstart (B33), browser playground, GGUF / CoreML / safetensors export.

The single-machine ↔ distributed boundary

What one Mac can do, what it can’t, and why — the line to own. Theory: advanced-llm-training (ZeRO/FSDP/3D parallelism), advanced-llm-inference §15–16 (disaggregation, TP/EP).

Case studies to learn from (boundary-mapping, not detours):

• Prime Intellect — globally distributed RL training (INTELLECT-2/3, 32B→100B+); training tolerates latency, so the swarm works here. primeintellect.ai
• teale / Petals — decentralized inference; the sharded-swarm dies on per-token latency, the whole-model-marketplace survives. teale.com
• oMLX — single-Mac serving done right (RAM↔SSD KV, continuous batching). omlx.ai
• Why decode resists distribution: memory-bandwidth-bound + sequential → network round-trips dominate. (advanced-llm-inference §1.)

Buildable hands-on (you don’t need a cluster to learn the primitives) — DP PoC ran 2026-06-17: MLX ships mx.distributed (all_sum / all_gather / sum_scatter / send / recv) + mlx.launch. scripts/archive/dist_dp_poc.py ran data-parallel all-reduce across n=2/4 ranks on one Mac and proved replicas stay bit-identical (per-rank param checksums matched; effective batch scaled 64→128→256). So all-reduce, sharded data, and lockstep replicas are learned firsthand; next rung = toy ZeRO (shard optimizer state via sum_scatter, re-gather with all_gather). The scale needs a cluster, the concepts don’t. (n=1 baseline hits a launcher JSON-init bug — minor.)

Learning gaps — the ⬜ shortlist (what to learn/build next)

1. Inference depth — batching + KV-SSD paging + prefix cache (B34); read the PagedAttention + FlashAttention papers, then build the batched eval-runtime.
2. RL/reward — close the full GRPO loop on a small model; understand reward modeling + over-optimization; RL environments as reward functions.
3. Distributed boundary — read the HF Ultra-Scale Playbook end-to-end; be able to whiteboard ZeRO stages, 3D parallelism, and why decode won’t shard.
4. Compression — logit-level distillation; fp8; the cost/quality frontier.
5. Agents — trajectory recording (B22) + agent-eval protocol (B23).

Next up — direction chosen 2026-06-17

Agentic tool-calling is now mined deep: frontier-parity distillation (4B 58→100, beats Gemma-12B)

• the teacher-free self-improvement loop scaffolded (self-improving-agents, journey §8, retention PRD). Diminishing product returns there → pivoting for breadth of learning. Four candidate areas, all queued (matter of time — we do the others in turn):

1. Single-machine ↔ distributed boundary [CHOSEN] — §boundary above; buildable via mx.distributed
   • mlx.launch (data-parallel + toy ZeRO across processes on one Mac). The named thesis gap; positions for scale.
2. Interpretability — §5; open the box on the distilled 4B (where the agentic skill lives; the mechanism of the measured negative transfer — journey §8.4–8.5).
3. Vision-language — §6; qwen3-vl-2b already local; a Pace pillar.
4. Mac-local serving systems — §3; continuous batching + KV-cache + speculative decoding (also ~10× the ReST rollouts — fixes the throughput bottleneck this session hit).

Step-back taken 2026-06-17 (inventory + ROI menu): the agentic-tool-calling lane is mined deep — file-ops saturated at frontier, breadth is a long-tail grind (+5pp from ReST), and shipping into Pace needs re-distillation on its action surface, not a wire-up. Verified this session: ReST self-improvement (breadth 65%), Apple on-device floor (can’t ground actions), VibeThinker reasoning (GSM8K 100%) + its training recipe, and the distributed DP PoC. ROI menu ranked; owner’s call pending. Diminishing product returns on agentic → next move is either the cheap specialist-merge verdict, a real re-distill-for-Pace project, or a pivot (distributed/serving) for fresh learning.

Living doc — correct coverage markers as reality changes. New learning lands as a focused page elsewhere in docs/learn/; this map just indexes + tracks.