Every markdown file in docs/ rendered as a page —
263 docs in total. The markdown source lives in the repo at
github.com/sarthak-fleet/tinygpt/tree/main/docs.
A "where things moved" table for anything you can't find lives at
/docs/MAP.
tinygpt agent <model> --tools tools.json is the product surface for the on-device agent SLM factory. The CLI loads a model (browser-trained .tinygpt, HuggingFace dir, or finetuned specialist), read…
Why async tool dispatch in AgentLoop doesn't justify the work at current bottleneck shape. Logged so the decision doesn't get re-litigated.
After ~70 techniques shipped across the project, this doc is the honest reckoning. Each entry: what it claimed, what we measured, and
the existing in-process MLX inference path. This is the "smoke test" that proves the scaffold produces real numbers; it is not a publishable benchmark (n=5, no energy metrics, tiny model).*
"most powerful inference engine for modern transformers" claim; nothing about that claim is credible without a reproducible measurement frame.*
How a Mac-trained BPE checkpoint gets a row on the browser leaderboard without the leaderboard ever needing to know what "BPE" means.
Phase 4–5. Move training/inference into the browser after the Python reference is correct. Order: PyTorch reference → WASM CPU → WebGPU.
Exhaustive map of input modalities tinygpt supports and what operations (train, distill, tune, quantize, etc.) are available for each. Honest status per cell — what's shipped, what's partial, what's only roadmapped.
Every architectural claim in TinyGPT's code + docs traces back to a primary source here. If a claim doesn't have a citation in this file, treat it as informed-opinion-not-evidence and challenge it.
Date: 2026-05-30 Branch: worktree agent-acdbc93fc70df3aed Target machine: M3 Max, 36 GB RAM, macOS 25.5 Build: xcodebuild -scheme tinygpt -configuration Release (LLVM optimisation on)
Reliable JSON output is the most common production failure mode for small (1-3B) LLMs in agent loops. Even when the model "knows" how to produce JSON, its sampler picks a wrong byte ~1 in N times —…
Status: implemented, measured, four items shipped. Numbers in this doc were recorded against the worktree binary built at the same SHA as the diff that introduced the items. Comparison baseline is …
Status: research + recommendations. No code shipped. Numbers are estimates unless explicitly called out as measured.
Two complementary data-pipeline regularisers for the Mac trainer:
Live snapshot of ~/.cache/tinygpt/datasets/ (not checked into the repo). Last refreshed 2026-06-17. Sizes are directory totals; row counts are from decoded JSONLs where available, otherwise marked …
Reference doc for every dataset wired into tinygpt — registry entries, what they're for, how to pull them, what the records look like after conversion, and known gotchas (gated datasets, parquet decode).
This is the meta-log: not the lessons learned, not the open questions, but the moments the direction actually changed. Across one long working session on TinyGPT, the project drifted, snapped, doub…
The browser app in browser/ is a Vite static build — no server, no API. It loads a pre-compiled WASM module from browser/public/, so the deploy environment does not need Emscripten; it only needs N…
tinygpt train --seed <UInt64> now seeds both of TinyGPT's two randomness surfaces:
Distillation is the post-training technique where a SMALL model ("student") is trained to match a LARGER model's ("teacher's") output distribution on a corpus. The result: students that significant…
Started 2026-06-11 after the "miner stopping 30 min before the diamond" check. Closed 2026-06-12 with the full table below.
ES is a finite-difference approach to optimisation: at every step, we sample K random perturbations of the current parameters, evaluate each perturbed model on a shared batch, and update the parame…
Implementation log for the second half of task #47. Forward landed last session (webgpu/attention_fa2.wgsl + tests/test_fa2_parity.mjs + docs/fa2_forward_notes.md); this is the runway for backward.
Implementation log for task #47 (FA2 in WGSL), forward-only. Backward will be a separate session — see "What's still needed" at the bottom.
Verification results absorbed into docs/PLAN.md §1. Kept here only as the audit's historical anchor.
A backlog of concrete features for the browser TinyGPT, drawn from the famous educational GPT projects worth learning from. Source repos are filtered to
End-to-end first specialist training on M5 Pro. What worked, what didn't, what was unblocked, and what to try next.
The browser playground ships v1 of the gallery: four bundled .bin checkpoints (Shakespeare, TinyStories, Python, Q&A chat) served from browser/public/gallery/. This doc lays out the next two iterat…
This page documents the five tinygpt train flags introduced in the Tier-2 stability batch:
tinygpt fetch-github pulls structured training data from GitHub for the code-specialist agent track (debugger, reviewer, commit-message generator). It is the GitHub-side counterpart to tinygpt down…
This doc captures the measurements from wiring custom gradient checkpointing into the Mac training path (Tier 1.7 of the single-machine roadmap). The numbers below were collected on an M5 Pro with …
Status: PRD (pre-experiment). Owner: Sarthak. Date: 2026-06-12.
tinygpt download-dataset and tinygpt list-datasets give the on-device agent-factory direct access to HuggingFace's 100k+ dataset hub. This document covers how to use them, how the format adapter wo…
This is the external learning track for TinyGPT. Use it after the repo-local archive/learning_roadmap.md: CS336 is the spine, and company docs/blogs are the applied case studies.
Two interpretability surfaces ship in the browser playground:
Date: 2026-05-30 Status: implemented (worktree), unmerged Target HEAD: 645c2f4 (main)
The TinyGPT leaderboard at /leaderboard.html ranks small, browser-runnable language models on a curated set of benchmarks. Three properties are non-negotiable:
You can write code. You have never built or trained a neural network. This guide takes you, in order, through everything in this repository until none of it is a black box.
This doc describes how tinygpt plugs into EleutherAI's lm-evaluation-harness, the canonical eval framework behind the HuggingFace Open LLM Leaderboard. With the wiring in this commit you can run He…
Phase 3. LoRA adapts an already-trained, frozen base model by training small low-rank matrices inside selected linear layers.
The browser playground is the on-ramp. The Mac app is the depth: same architecture, same .tinygpt file format, ~20-30× the training throughput. This doc translates roadmap lever 20 into a concrete …
This restructure split, merged, and archived a few docs. Use this table to find where the content of any old path now lives.
What fits on a 48 GB Mac for training is dominated by four memory costs: weights, optimizer state, gradients, and per-step activations. This guide explains each in concrete numbers and the levers w…
Phase 1–2. Build a tiny GPT-style causal language model. First goal is
MoE replaces a transformer block's single dense MLP with N "expert" MLPs plus a learned router that picks the top-K experts for each token. The result: a model with ~N× the parameter capacity of th…
Standard language-model training predicts ONE token ahead per position. Multi-Token Prediction (MTP) predicts H tokens ahead per position simultaneously, using H output heads that share the same hi…
This is the active queue. It intentionally ignores most historical PRDs.
How a documented scaffold became a verified, end-to-end implementation of the modern LLM stack: a GPT trained from scratch, adapted with LoRA, ported to hand-written WebAssembly kernels, accelerate…
Audience: an ML-curious engineer who can read C++/WGSL but hasn't derived the trick before. This doc walks through what "online softmax" means, why it matters for attention specifically, and where …
TinyGPT ships five optimizers, selectable on tinygpt train, sft, and dpo via --optimizer {adamw|lion|sophia|muon|adafactor}. Default is adamw (preserves backward compatibility — pre-existing script…
tinygpt (the factory) refocuses on local-LLM research. This doc freezes the state of everything tinygpt owes or has delivered to Pace (the consumer), so either repo can pick up any thread without r…
action). Sub-head: "CodeVetter is a desktop review cockpit for the diffs your agent ships. Catch what Cursor, Claude Code, and Devin missed — vulnerabilities, regressions, and silent drift. Runs en…
Canonical inventory of all models Pace needs (goal set 2026-06-10: all roles decided with runnable artifacts by end of day). One row per role; this doc is the single source of truth — update it whe…
A bundle of parameter-efficient fine-tuning (PEFT) flavours, all wired into the existing finetune / sft / dpo commands as opt-in flags. The default behaviour (vanilla LoRA r=4 α=8 on q/v projection…
Two-item perf + correctness audit, scope-limited to:
A working document for the in-flight push to give users on the latest Chrome every GPU acceleration the platform exposes — without ever degrading model quality. Started during the HN-launch prep th…
A working document for the "best in market" target. Tonight's session landed KV-cached sampling (~2× sustained), 4-bit palettization (6× smaller files), and the ANE conversion path. This doc survey…
How fast TinyGPT trains, what has been done to speed it up, and what is left. All numbers are from an Apple M5 Pro laptop.
This is the stop point for planner-model churn. Starting tomorrow, planner work should be integration, routing, latency, and daily-use fixes, not another base-model search.
How TinyGPT ensures every accelerated code path preserves model quality before it activates for the user. The non-negotiable rule from the opportunistic-acceleration scope (see docs/perf_quest.md a…
Consolidated into docs/PLAN.md §1 (shipped). Headline metrics live there.
Generated: 2026-07-02T00:00:00.000Z
Pruning is the post-training technique where you remove parts of a trained model and accept a small quality hit in exchange for a smaller model. There are two flavours here, with very different pay…
A chronological reconstruction of the working session — the questions asked, what shipped in response. Format is question-then-answer, with an Outcome line per entry pointing at what actually shipp…
Three loose pieces of the build/test infrastructure shipped together so the next agent inherits a CI floor that catches the obvious regressions before they reach main.
This drop ships four new quantization features on top of the existing AWQReader (HF AWQ safetensors loader) and HQQ (storage-only Half-Quadratic Quantization) that already live in the tree:
quickstart turns a data file into a trained, runnable specialist on your Mac with one command and zero ML knowledge: it inspects the data, auto-picks a base from the gallery, infers a LoRA recipe, …
Written 2026-06-11 after the clarify-v1 experiment failed and the small-specialist training bet closed. This doc replaces no other; it records what we learned at high cost so future work doesn't re…
Consolidated into docs/PLAN.md §3 (TODO). Tier A/B/C/D ordering preserved.
This document captures a single long working session on TinyGPT — what was believed coming in, what the contradictions turned out to be, what got shipped, and what the transferable lessons were. Th…
Multi-hour session that took tinygpt from "Wave 2.5 stalled" to "every CLI verified + first specialist trained end-to-end + ROI-ordered backlog." What shipped, what was learned, what to pick up next.
Two joint-trained draft-head architectures that bolt onto a frozen base LM and propose multiple tokens per base-forward, verified in a single extra base forward pass. Implemented in the native-Mac …
Two changes that let tinygpt sample survive arbitrarily long generations on a memory-bounded Mac:
The technical material this session touched, organized so you can pick what to learn at your own pace. The chapters are thematic, not chronological. Each one answers the same two questions in a dif…
tinygpt targets EXEMPLARY test coverage across all three runtimes: Swift (native Mac), Python (reference impl), and TypeScript (browser).
Single source of truth for what's shipped, skipped, and still to build. Consolidated from docs/roadmap/*, docs/progress.md, docs/backlog.md, docs/feature_audit_2026_05_31.md, and docs/roadmap/recent_research.md (paper catalogue). Replaces them as the canonical reference; the older docs are now pointer stubs or archived under docs/archive/.
Design + training recipe for tinygpt's tiny encoder model that picks which tool a user query needs, before the full LM forward pass.
tinygpt train --depth N derives every pretrain hyperparameter from one number — architecture and the learning-rate / batch / step schedule — following the nanochat surface (karpathy/nanochat) and t…
tinygpt exposes an Ollama-compatible HTTP surface so it drops straight into Continue.dev, Cline, and Aider configs as a local provider. Setup + caveats.
This document records the v9 baseline against each of the six v11 ship-gate dimensions (pace-planner-v11-ship-gate.md). Frozen at commit time. Not for revision — v11 progress measured against these…
This doc captures the validation runs and workflows that exercise the features shipped in Phases 1-10. The goal isn't peer-review rigour — it's enough evidence that each surface RUNS end-to-end on …
Researched 2026-06-10 via two web sweeps (platform + product). Sources at bottom; unverified items marked. macOS 27 is "Golden Gate", fall 2026.
This doc captures the implementation and smoke-test measurements for YOCO (Lin et al., 2024) in the Mac training path (Tier 3.8 of the single-machine roadmap). All numbers below were collected on a…
Consolidated into docs/PLAN.md.
Verified URLs, sizes, and licenses for everything in the pretrain → SFT → DPO pipeline. Picked for "this fits on a 48 GB Mac" or "we can stream a slice."
This doc supersedes the rough Tier 5 sketch with a sharper product vision. The project is now:
Consolidated into docs/PLAN.md §2.
Consolidated into docs/PLAN.md.
Consolidated into docs/PLAN.md. Several markers in the old version were stale; the merged doc is verified against code.
Consolidated into docs/PLAN.md. Most phases shipped; remainder in §3.
Consolidated into docs/PLAN.md §3.
Consolidated into docs/PLAN.md.
Consolidated into docs/PLAN.md.
Consolidated into docs/PLAN.md.
Consolidated into docs/PLAN.md §2.
Consolidated into docs/PLAN.md §3 Tier 5.
A modern useful language model is the product of three distinct training phases, each with its own dataset shape, loss function, and goal. This section walks the whole pipeline as it exists in Tiny…
The third of three training phases. See pretrain.md and sft.md for what comes before.
The first of three training phases. See docs/training/index.md for the overview, docs/training/sft.md and docs/training/dpo.md for what comes next.
The second of three training phases. See pretrain.md for what comes before and dpo.md for what comes after.
Read-and-evaluate write-up on the modded-nanogpt speedrun's "cross-stream attention" trick; decide whether to adopt for TinyGPT.
How tinygpt scores Indic-language ability — MILU multi-choice across 11 langs, IndicGenBench XQuAD extractive QA. The Wave 4 gate before claiming Hindi support.
between my Jan 2026 knowledge cutoff and current state. Includes URLs for verification.*
The Mac is 10-20× faster than the realtime/interaction-model targets across every model size we have. cider's prefill win is real but immaterial at current scales; defer until there's a 3B speciali…
One artifact that cross-cuts decode speed, BFCL, τ-bench, and Pace unhappy-paths — the publication-shape view we're missing.
between my Jan 2026 knowledge cutoff and current state. Includes URLs.*
the engineering vs MLX-Swift / CoreML / Apple defaults?
should know about before training specialists?
The conversation, condensed. Long replies have been squeezed to their actionable conclusion; emoji-free; some early phases reconstructed from compressed records. What follows is a chat-style record…
Phase 9 — and ongoing. Know what the model is actually doing.
The path to learning the whole stack without drowning in complexity. This is the curriculum; model_guide.md, lora_guide.md, and browser_notes.md are the implementation detail.
What didn't work, what surprised me, and what I'd carry forward. The kernel optimisations are documented in docs/performance.md and the milestones in docs/archive/status.md. This is the meta layer:…
User decision (this session): hold these four options and focus on the HF-compat capabilities (SwiGLU + RoPE + GQA + BPE tokenizer) first. Once those land, the Mac app can load any modern open-weig…
This doc closes out the remaining Phase 9 (quantization) and Phase 10 (architecture menu) items. For each: what's shipped today, and for the items not yet shipped, what's needed to land them.
The 2024-2026-era body of work that informed Tier 1-3. Each entry is just enough context to know whether to dig in.
TinyGPT will ship as two binaries: a browser playground (Chromium + M-series targeted) and a native macOS app (M-series only). They are separate implementations of the same model, optimized for dif…
A review-oriented snapshot of where TinyGPT stands. The detailed docs are linked at the bottom; this page is the map.
The TinyGPT playground has a small interpretability lever bolted onto the Sample card: click any byte the model just generated and see the two things the model "actually knew" at that position — th…
The eval protocol exists to prevent training against noise.
Hugging Face Hub is the default public artifact store for TinyGPT factory outputs.
TinyGPT's active product is the specialist factory.
A shipped specialist needs a small, inspectable package. The package metadata is committed; the large model or adapter artifact usually stays under ~/.cache/tinygpt/models/ or another local/cache p…
TinyGPT's active center is a Mac-local specialist factory, not a generic fine-tuning notebook.
Public artifacts are first-class factory outputs. They are different from local run files: a public artifact should be understandable outside this repo, have a small committed metadata surface, and…
Start here for active TinyGPT work.
Every factory run should end with a report, even if the decision is reject.
Each real factory run should write a local run directory:
Senior/staff topics spanning modern transformer architecture, the linear-algebra↔transformers whiteboard, RAG, agents, evaluation (LLM-as-judge, perplexity, contamination), ML system-design rounds, and classic-ML depth — each with the best external source and repo anchor.
Senior/staff inference-optimization interview topics — KV cache & paging, batching, speculative decoding, quantization, attention kernels/variants, long context, serving architecture — mapped to the best external source and to where this repo touches it.
Senior/staff interview topics for LLM training at scale (parallelism, precision, MoE, data) and post-training (RLHF/DPO/GRPO/reward modeling/distillation), each mapped to the best external source and to where this repo touches it.
The L1/L2/L3 framing for agent context engineering — what we stole from the Shortcut vertical-agents essay and where each steal lives in this repo.
Status: research landed 2026-06-07 Supports: docs/prds/factory-planner-v7-tools-in-prompt.md
What Apple's FoundationModels framework + on-device model give you, the bridge we built to score it on our gates, the measured verdict (can't ground actions; 4096-token context can't hold a tool catalog; not faster), and the strategic decision — use it as a free floor for routing, never as a dependency.
Patterns lifted from castform.com (RL fine-tune SaaS) into TinyGPT's specialist-training surface. Sibling page to docs/learn/agent-context-hierarchy.md.
A 2026 map of the products around "make a model good at your task" and "is my agent any good," with the Mac-first whitespace called out. Evidence behind docs/sessions/2026-06-13-market-landscape-mac-first.md.
A 7-session curriculum that takes you from "what's a neural net" to "I can read any modern ML paper without hand-waving."
Three transferable training methods from the VibeThinker papers — Diversity-Exploring Distillation, MaxEnt-Guided Policy Optimization (MGPO), and specialist weight-merging — mapped to where each upgrades our own self-improvement loop. The "how a 3B matches flagship reasoning" recipe, and why one of its steps is the direct antidote to the negative transfer we measured.
Status: v1 research synthesis landed 2026-06-07 Supports: docs/prds/factory-ane-inference-pace.md
v8 was trained on v5's 248-row corpus + 59 hand-crafted examples targeting v2's failure modes (semantic disambiguation, multi-element reasoning, abstract reference). Same hyperparameters as v5/v6: …
endpoint that runs zero model inference. Every Pace LoRA we trained scores ≤ this baseline. The fixtures test framework, not model.
Curated articles, papers, and projects relevant to TinyGPT. Each entry is one-sentence-what + one-sentence-why-for-us + link, per the docs preference for leaning on authoritative external sources r…
A running log of questions, hunches, "wait, why does anyone do it this way?" moments, and tangents that come up during the curriculum sessions.
TinyGPT's learning corpus — a ground-up curriculum from "what's a neural net" to modern training mechanics, plus reference + research notes.
For a 12-layer Huge model: ~108 matmuls in the body + ~1 input lookup + 1 output projection = ~110 matmuls per next-token prediction.
Empirical diagnostic of the ANECCompile / runtime failure on the Qwen3-0.6B stateful CoreML path.
The learning + build agenda for mastering Mac-local AI. Every capability buildable on a single Mac, annotated with fleet coverage (built / partial / to-learn), plus an explicit single-machine ↔ distributed boundary. The spine for "learn everything + build everything buildable on this Mac."
The single most useful lens on this project — which parts compute the model's function (irreducible math, the correctness oracle) vs which parts only make that function cheaper/faster (the optimization layer). Plus the one number that polices the boundary: loss drift.
The architectural distinction between a fine-tuned model and the agent that runs it, and what "making a good model into an agent" actually requires. Mapped onto this repo.
Documented 2026-06-08 from huggingface/transformers/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py. Closes research tasks #277 and #278.
Parked blueprint. The fleet's AI game (autonomous characters in a world) is already a multi-agent RL environment — the scarcest RL ingredient. This is the plan to turn it into a self-improving-agents testbed on the Mac, and the staged roadmap toward multi-agent dynamics. Revisit after distillation + the first GRPO loop.
input → same output, every time.
A neural net is, at the bottom, a function:
Before we can search for "good" values, we need to define what good means. The whole search machinery is built on this one definition.
What can this setup not do, and how do we fix it?
When people say "supervised learning," "neural net," "gradient descent," "RL," "tree regression" — they're naming things from three different axes that are mostly independent and freely combine. Th…
You just watched a 22M-param model learn English structure but flunk basic factual recall. The natural question: what changes at 100M, at 1B, at 100B? And: is bigger the only knob?
Sessions 1–5 covered what a neural net IS and what it learns. But we've been hand-waving over a key question: the model is a function on numbers. Where do the numbers come from when the input is text?
A language model's job is: given the text so far, predict the next token. An agent's job is: given the situation, pick an action. Both are "functions on data" — the math we built in Sessions 1–5 st…
Session 2 covered gradient descent in the abstract: pick parameters, compute loss, take a step. Sessions 3–7 covered what the model IS, what data goes in, what paradigms exist. But the actual machi…
SFT tricks, RL, eval, on-device serving — distilled to what's stealable on a Mac.
Voice-pipeline latency, WER, speech-to-speech, fine-tuning debugging, feature selection, queues vs websockets, FSDP2 — each mapped to the best external source and to where this codebase (or Pace) actually does it.
The mental model you need to read TinyGPT's WGSL compute kernels and explain them to another engineer — device/queue, pipeline, dispatch, workgroups, invocations, bind groups, the memory hierarchy — then how our matmul + attention shaders map onto it.
metric, an honest size curve, and a distillation result that closes most of the gap.
ANE/CoreML work produced useful measurements and negative results. It is parked until a shipped specialist needs battery/perf optimization.
Broad Mac app polish is parked. The app should not become the center before the factory loop is proven.
The browser playground and WebGPU path are successful completed assets. They are parked for active factory work.
These lanes are not abandoned. They are paused so the factory proof can finish.
These are learning-rich but not the current factory proof.
Vision-language work is parked for the factory proof.
Train a 22M-class model with reasoning-style supervision (GRPO / DAPO on verifiable rewards; SFT-on-traces as the warm-up), publish the
Run the Snell et al. 2024 methodology end-to-end at 22M: vary the test-time compute budget (N samples for Best-of-N, M tokens in CoT, K depth in tree search) and measure the quality-vs-FLOPs curve …
Train a LLaVA-style VL model from scratch on M5 Pro: pre-trained ViT encoder (frozen first cut) + small projector + small-from-scratch TinyGPT decoder + cross-attention or projector. Evaluate on Te…
Train a small (22M-class) diffusion language model from scratch on the masked-denoising objective (Austin et al. 2021; DiffusionBERT, Sahoo et al. 2024). Different generation paradigm than autoregr…
Build the missing-upstream Metal kernels (sparse top-K routing, scatter_add equivalent, expert-parallel matmul gather) so MoE in TinyGPT can run hard routing — actually skipping inactive experts' c…
Train a from-scratch autoregressive decoder over discrete audio tokens (EnCodec codebook IDs) conditioned on text, à la VALL-E / MusicGen. Single-speaker (LJSpeech), recognizable-but-not-natural sp…
Build the smallest useful "explainer video" system: prompt or document → script → storyboard DSL → deterministic SVG/Canvas renderer → MP4 with captions + voiceover. NOT a competitor to Sora/Runway…
Ship one trained specialist that beats Pace's current 0-shot floor (qwen3-4b-instruct-2507) on BFCL average by ≥ 3pp on a Mac-runnable artifact (≤ 8 GB on disk). Validates the platform's north-star…
Run the factory recipe shape against SQL to validate that the platform is not accidentally tied to a tool-calling target. Same factory loop, different data and eval.
Ship tinygpt quality-classify <corpus.txt> that scores every document on an educational-quality axis, writes the per-doc scores to a sidecar, and filters to a top-X% subset for downstream pretraini…
Add --lr-schedule wsd to tinygpt train (tinygpt finetune / sft / distill get it for free since they share the optimizer wrapper) with three flags: --warmup N (steps to peak LR), --stable-frac F (fr…
Add a grad-norm tracker that triggers an automatic `--resume <step-K>
Replay the shipped interp tools (tinygpt sae, tinygpt memit, tinygpt rome, tinygpt patch, tinygpt causal-trace) across the multi-checkpoint history a single tinygpt train run produces, so we can se…
Wire --draft-model <path> into tinygpt serve and tinygpt sample so a small "draft" model proposes K tokens per step and the target model verifies them in one parallel forward. Expect ~1.6–2× decode…
Add --llrd γ to tinygpt sft (and the DPO family for symmetry). For γ < 1, each transformer block's LR is multiplied by γ^(depth_from_top), so the embedding + lower blocks see exponentially smaller …
Verify, on TinyGPT's actual MLX path, the Apple ML Research, 2026 claim that the M5 Neural Accelerator delivers 3.5–4× faster prefill vs M4 on the same workload. Bump mlx-swift to the latest (0.31.…
Add tinygpt sae export --format saelens <input.sae> --out <dir> that converts our shipped .sae sidecar format to the SAELens on-disk format (Neuronpedia ingest target). One-way export only — we kee…
tinygpt train --depth 12 ... should auto-derive d_model, n_heads, d_mlp, peak_lr, batch_size, and total_steps from compute- optimal scaling laws. UX win — most users don't want to tune 6 hyperparam…
Add tinygpt sae --group-layers SPEC so a single SAE is trained on the concatenated activations of a contiguous block of layers — e.g. --group-layers "0-3,4-7,8-11,12-15" trains 4 SAEs for a 16-laye…
Bundle the four router items from PLAN.md Tier B into one coordinated PRD because they are sequentially dependent on each other:
Replace the current "hand-wave a 50/30/20 code/web/math split" pattern with a small AutoMixer-style ratio search. Train ~6–12 short proxy runs across candidate ratios, score them on a fixed capabil…
Replace single-shot agent eval runs with the Poolside-class protocol: each task is run K times under fixed budget constraints, results are averaged with confidence intervals, and the exact resource…
Train a Mac-runnable specialist that takes (query, long_context) and returns the subset of sentences relevant to the query — i.e. extractive compression. Submit to the public ScaleDown Challenge le…
Today, tinygpt serve --tools <catalog.json> injects every tool's full JSON schema into the system prompt of every request via ServeToolsSpec.systemPrompt() (DynamicGrammar.swift). For a catalog of …
Ship a typed CompositeReward abstraction: a reward is a bag of named scalar dimensions + weights → total. Every dimension is independently logged, plotted, and inspectable. DPO / ES / GRPO (5.1) al…
browser/public/gallery/manifest.json so it lists not just the browser-loadable from-scratch .bin models but also the
Ship tinygpt eval-gate — a single command that runs a project's declared eval suites against a model + adapter, compares to a stored baseline, and exits non-zero when any suite regresses past a thr…
tinygpt quickstart <data.jsonl> takes a user's task data and walks them to a trained, evaluated, runnable specialist on their Mac with one command and zero prior knowledge — auto-picks a sensible b…
Make tinyGPT's eval harness fast and scalable by stealing the two oMLX techniques that fit eval's workload — continuous batching and a
Prove the local-only specialist agent thesis on one concrete vertical: a code reviewer that runs entirely on the user's Mac on a tinygpt fine-tuned 12B model, with zero cloud dependency, and beats …
Train a specialist to emit {"defer_to_cloud": true, "reason": "..."} when it shouldn't answer itself — too uncertain, off-domain, or explicitly destructive — instead of fabricating. Today's tinygpt…
The product-shaped artifact that the platform's pitch hinges on: a Mac app the user can open, point at their data, pick a base model, train a LoRA, and run inference on the specialist they just tra…
Train a multilingual SFT/LoRA specialist on a Sarvam-Edge or Airavata base (both are Indic-LLM bases optimized for English + 10+ Indian languages) for a Mac-runnable Indic-capable agent. Ship gate:…
Measure energy-per-generated-token across the model zoo and across training runs. Surface a j_per_token column on the SLM leaderboard and a per-epoch number on training runs. Answers "what does it …
Ship a browser page (/train-viewer.astro) that drag-drops a tinygpt train run-history directory (or watches it live via OPFS on the same machine) and renders the canonical live-training charts: los…
DoRA already trains in-session under tinygpt sft (default PEFT variant since 2026-05; PeftVariants.swift). It does NOT yet persist to disk in a reusable format — the adapter lives only in the runni…
tinygpt train-extractor (the mini-router-trainer that ships the "intent + tool" classifier on top of the residual stream) currently assumes a byte-level tokenizer. Bases the user actually wants to …
Add scripts/bench_decode_thermal.py that runs the existing decode bench in a sustained loop for N minutes (default 30), captures per-window p99 tok/s + die temperature + power draw via powermetrics…
Add tinygpt train --replay-step N --replay-from <ckpt> that runs exactly step N of an earlier training run, deterministically. The output is a logged forward + backward trace identical to what the …
Measure and preserve a small model's GENERAL intelligence when we specialize it (tool-calling, style, persona). This session proved specialization erodes breadth — a retention eval battery + retention techniques, run on every fine-tune. Future work; captured for later.
promote it. The model gets better from real usage — and privacy isn't a compliance checkbox, it's the default.
Ship tinygpt eval-scaledown <model> that runs the official ScaleBench harness against a TinyGPT-loaded model and emits results in the shared E0 EvalCompare.Row schema. B25 specialist needs this for…
Tracks the "make the factory complete" thrust spawned 2026-06-07.
Owner approved the previously gated v7 work. The first implementation slice landed:
The VLM elf shipped M1+M2+M3 cleanly. At M3 (and after the leverage-first gate added to the PRD mid-flight), the elf identified that UI-Venus-1.5-2B is the best open-weights student base — not the …
Context already shipped (M1-M3):
M1-M3 are shipped. The M4 architecture decision is now made in factory-vision-m4-architecture-decision.md: Option A, full Qwen3-VL port with UI-Venus-1.5-2B as base.
Turn the fleet's AI game (autonomous characters in a world) into a Mac-local RL environment and train a self-improving NPC with GRPO. Environments are the scarcest RL ingredient — we own a live one. The parked trigger (distillation closed + a working GRPO loop) is now MET.
Apply GEPA (Genetic-Pareto Prompt Evolution, ICLR 2026 Oral) to automate the system-prompt + action-description iteration loop. Stop hand-tuning prompts; let evolutionary search find the Pareto-front.
The PRDs below can't be finished-and-verified in a CPU sandbox: they need a GPU (training is the deliverable), special hardware, network installs, or are multi-week from-scratch model builds. The v…
Pit Mac-local models against each other and against frontier models in turn-based strategy games, then self-play-RL a local model until it beats a frontier model playing zero-shot. The match win/lose IS a verifiable reward — RLVR with no hand-authored golds or checker.
Status: IN PROGRESS (2026-06-10) — Phase A SHIPPED, Phase B in flight. Spawned out of the 2026-06-09 research sweep + anemll-vs-M8 decision. Target: 17 tok/s → ~30 tok/s on the existing Qwen3-0.6B …
A Mac-local, frontier-gated, STATEFUL multi-turn eval — the single biggest blind spot in our tool-calling work. Single-turn numbers (88.7 for our best 4B) overstate agentic ability; small models are known to cliff hard on multi-turn. This is the prerequisite for judging any model as an agent/planner.
v1 through v10 of the Pace planner cycled because we trained → measured → argued whether the version was better. No fixed bar, no fixed eval, no immutable record of pass criteria. The v8 "73.3% non…
Produce ~450 training rows that teach a Qwen3-0.6B planner to emit three new intent classes:
Status: PRD (2026-06-10). Triggered by a real user moment: bulk-downloading files via the Claude browser extension was painfully slow. The standing rule (memory pace-does-it-not-claude): the delive…
This file is the working priority map for every PRD currently on disk.
Status: PRD (2026-06-10). Phase-3 centerpiece of the roadmap (specialist close-out → VLM/Pace → this). Implements the 2026-06-10 strategic shift: focus moves from training small models to fine-tuni…
Status: PRD (2026-06-09). Phase 1 SHIPPED 2026-06-10 (serve --quantize); native packed load (the design below) still open.
Active work starts in ../NEXT.md, not here.
Close the act -> score -> learn -> curriculum loop so a Mac-local agent improves itself with NO teacher, using a verifiable reward. The smallest proof: a teacher-free ReST loop on the file-ops env that raises pass-rate round over round. The compounding engine: an automatic curriculum that keeps proposing tasks at the edge of the agent's ability.
Owner approval for heavy work was granted. LM Studio is reachable at http://127.0.0.1:1234/v1/models and includes qwen/qwen3-30b-a3b.
Historical, code-cross-checked status of every active PRD in docs/prds/. Each verdict was verified against native-mac/Sources/, scripts/, evals/, and browser/src/ — not the PRD's own status: frontm…
Status: positioning doc (2026-06-10). Written as the specialist track closes (v11 = final 0.6B planner run, verdict pending) and focus shifts to fine-tuning/distilling larger open models. This doc …
Status: PRD (2026-06-09). The 2026-06-09 research sweep flagged Qwen3-VL-2B as a competitor to our UI-Venus-1.5-2B port target. This A/B decides which one we actually port to MLX-Swift / CoreML / A…
Stop hand-waving "50/30/20 code/web/math". tinygpt automix searches the ratio: sample candidate mixes, score each with a short proxy train run, fit a quadratic surrogate, and propose the next mix b…
and outputs an extractive compressed document — a subset of the original sentences that preserves the answer-relevant span — and submit to the ScaleDown.ai challenge.
Use B30 ahead of B29 to control single-hop / multi-hop / comparison ratios in the SFT corpus.
What this is not: a magic 70B replacement. The goal is a small specialist that stays in character on a narrow world better and faster than the base model.
What this is not: an editor plugin. Continue.dev and Aider already know how to call OpenAI-compatible local servers; TinyGPT only has to serve the model.
What this is not: automatic correctness. Pydantic validates shape; your eval still has to measure whether the values are right.
What this is not: a claim that a tiny model beats a frontier model on broad agent tasks. This is for narrow specialists: function calls, schemas, repo idioms, and other repeated local patterns.
Compress a big model's capability on your task into a small, cheap, local model on the Mac. This is the validated cost-compression lane — the lever that
(3.8B) on function-call accuracy while running 15-30× faster and using 150× less memory on the same Mac.
Gate a TinyGPT specialist in CI — `tinygpt eval-gate` exits non-zero when any declared eval suite regresses past threshold, on a self-hosted Mac runner so the model never leaves the device.
Use this when a Pace planner candidate needs an unhappy-path score that is publishable or comparable across runs. Model selection itself is locked in docs/planner-lock-2026-06-19.md; this recipe is…
B22 + B29 + B30 + tinygpt sft chained — the closed substrate→training loop, V1.
Use tinygpt export-mlx when a TinyGPT-trained artifact needs to leave the TinyGPT binary and be loaded from Python MLX, MLX-Swift, or another Mac-local tool.
The factory's first product-validating arc. Distill Pace's qwen3-30b-a3b planner into a small student. See docs/prds/specialist-pace-planner.md for full spec.
Copy-paste workflows for using TinyGPT specialists outside the core CLI.
be training the model and working on site projects. Before firing the long training run (N02), close every gap that would block scoring the resulting model — schema, harness, baseline comparison, m…
benchmarks." The real thesis is: a Mac app where individuals build a specialist for their specific task — bring data, pick a teacher, ship a fast/cheap/comparable-quality model. Distillation, fine-…
PRD: docs/prds/specialist-embedder.md
Recommendation: partial-proceed. Do not wrap vllm-mlx into tinygpt serve yet. Keep native serve as the default and as the eval/logprob backend. Treat vllm-mlx as a production-chat candidate after a…
real FC specialist. Find every gap.
(fine-tune SaaS) and "is my agent any good" (eval/observability). Where does a Mac-first SLM toolkit fit, and what is genuinely unowned? Triggered by the castform.com scan; researched the broader l…
Date: 2026-06-16. The arc, the decisions, and the transferable lessons. Numbers and mechanics live in tool-calling-frontier-parity.md §8.1–8.5; this is the meta-narrative.
Date: 2026-06-17. A deliberate pause to inventory the assets, name the wall, and rank what's next by ROI — before committing to another build. Narrative of the arc lives in the 2026-06-16 session d…
Date: 2026-06-17 (afternoon). Companion to the morning's step-back. Captures the strategic decision triggered by Vercel's Eve launch (blog, public preview June 2026).
LR decay) · Ship gate: +3pp on the Berkeley Function-Calling Leaderboard (BFCL) over the stock base.
Cookie-cuts the A1 recipe onto a second domain to prove platform generality.
Status: expanded 0.6B factory POC complete; next step is preference tuning or a public benchmark slice.