Small models.
Real evaluations.
Vertex AGI builds single-purpose language models distilled to do one job well, evaluates them exclusively on held-out data, and is training a mixture-of-experts model from scratch — in the open, on hardware anyone can buy.
Models shipped
Parameters in the from-scratch pretrain
Tokens of pretraining data
Through the pretrain run
Model family
Prism.
One job, done exceptionally.
Our specialist line — 9 models, each distilled to do exactly one narrow task well enough to replace a frontier-model call: titling, safety classification, creative writing, roleplay. Nothing general-purpose about them.
- Models
- 9
- Size range
- 0.6B–8B
- Disciplines
- 4
Model families
Four families, one philosophy: small, cheap, and good at exactly one thing.
Distilled by LoRA from frontier teacher models rather than trained from scratch — the from-scratch work lives separately, below.
Prism
Single-purpose specialists — Narrow models distilled to do exactly one job — titling, safety classification, creative writing, roleplay — well enough to replace a frontier-model call.
Amethyst
General-purpose chat — Broader conversational models, including tool-calling variants, built for everyday assistant use rather than one narrow task.
Copal
Agentic tool use — Models trained specifically to decide when and how to call a tool — not just chat, but act.
Aquamarine
Code specialist — A coding-distillation model in active training: generation, debugging, and refactoring across Python, JavaScript, TypeScript, Go, and SQL.
Research
A mixture-of-experts model, trained from zero
Not a fine-tune — randomly-initialized weights, a DeepSeek-style fine-grained MoE, training on a single 6GB consumer GPU. No cluster, no cloud credits.
Live — loss 4.7417–5.2442, ~48 tok/s
Total parameters
1,031.0M
~1.03B
Active parameters / token
394.0M
~394M
Hidden size
1,024
Layers
20
first 2 dense, rest MoE
Attention
16Q / 4KV
GQA, head dim 64
Context length
2,048
tokens
What we've learned
A few honest findings from this cycle.
Synthetic data teaches form, not capability
Narrow synthetic distillation reliably fixes formatting and behavior — a title-length spec, suppressed reasoning leakage, correct tool timing — but doesn't reliably add knowledge the base model lacked. Real, human-annotated data is what moved raw accuracy: Prism Safety's BeaverTails recall went from 13% to 79.5% only after mixing in real labeled data.
Base model choice can matter more than fine-tuning
An untuned Qwen3-1.7B outperformed multiple fine-tuned Gemma-based safety classifiers on held-out benchmarks. Picking the right foundation is sometimes a bigger lever than the training run itself.
Held-out evals are non-negotiable
Early evaluation passes that sampled from the same template bank used for training scored close to 100% — because the model had simply memorized the templates. Every eval we publish uses hand-authored or independently-sourced held-out data with verified zero overlap against training.
Small models plus consumer hardware go further than expected
Every fine-tune we've shipped — and the from-scratch 1B-parameter MoE pretrain — runs on Apple Silicon or a single 6GB gaming GPU. No cluster required.
Every model, every weight, on Hugging Face.
MLX and GGUF builds for every release — run them on Apple Silicon or via llama.cpp, LM Studio, and Ollama.