MenteE Code 3B v1: Training a Coding-Agent Model on 53,567 Trajectories for $0.13 (Full Report)
We fine-tuned Qwen2.5-Coder-3B-Instruct with LoRA on 53,567 execution-verified agent trajectories for $0.13 on one RTX 5090. Full report: loss curves, HumanEval results published with the regression included, root-cause analysis, and the v2 recipe.
MenteE Code 3B is our coding-agent model: a fine-tune built to live in a terminal, investigate a repository, edit files, run tests, and report honestly. This post is the full v1 training report. It includes the run configuration, the measured loss and accuracy, the benchmark, and the part most teams would cut: the regression we found, the root cause we traced it to, and the v2 recipe designed to fix it.
The thesis in one paragraph
A small mixture-of-experts model can behave like a model ten times its size on agentic coding. Qwen3-Coder-Next activates about 3B parameters per token and reports 70.6 on SWE-bench Verified, which demonstrates the pattern. The mechanisms that make it work split into two groups: the ones you inherit from a good base model (sparse knowledge storage, cheap long-context attention, native tool-call format) and the ones that can only come from your own data (execution-verified post-training and long-horizon recovery behavior). Our project is built around the second group. The dataset flywheel is the moat; the training script is a commodity.
The data: 53,567 verified trajectories
v1 trained on 53,567 schema-validated agent trajectories, deduplicated from 78,212 collected runs. Sources: SWE-agent, OpenHands, R2E-Gym and NVIDIA SWE-Zero subsets, plus runs from our own collector, which validates every row against a canonical JSON Schema before appending, and identity Q&A for persona calibration. The dataset file is append-only: never truncated, never rewritten, checked against the collector's own belief as it grows.
The run configuration, fully measured
That is the entire infrastructure requirement for v1: one consumer GPU and about thirteen cents. The claim is not that this is a frontier model. The claim is that the full loop, data collection, schema validation, training, evaluation and publishing, runs at a cost low enough to iterate weekly.
Evaluation: we publish this number on purpose
HumanEval pass@1, greedy decoding, 50 problems, both models on identical hardware with an identical harness, generated code executed against the official unit tests:
Yes: v1 scores well below its own base model, and we are publishing that. Failure analysis found that 24 of the 25 failures share one artifact: systematically doubled closing brackets. The cause is the v1 loss recipe, which applied loss to all trajectory roles, including noisy tool outputs, so the model learned to imitate tool noise alongside assistant code. It is a data-recipe issue with a known fix, not a capability ceiling. Two honest caveats: this is a 50-problem subset with roughly a ±14 point margin, so the numbers are directional rather than leaderboard-grade, and HumanEval is not the benchmark the thesis ultimately targets; SWE-bench Verified with a real agent harness is.
The v2 recipe
- Assistant-only loss — compute loss on assistant tokens only, so noisy tool outputs are context, not imitation targets.
- Identity oversampling — keep the persona stable while the coding behavior changes.
- LoRA r=16 — more adaptation capacity for the behavior correction.
The same run costs about thirteen cents, so the iteration loop is measured in days, not quarters.
What ships today
- Merged weights on Hugging Face (MenteEAI/MenteE-Code-3B), Apache-2.0, with 32K native and 131K YaRN context.
- GGUF builds for Ollama and llama.cpp at MenteEAI/MenteE-Code-3B-GGUF.
- OpenAI-compatible serving via vLLM, tested with OpenCode-class harnesses.
- The collector toolkit and full docs in github.com/MenteE-s/MenteE-Code-3b.
v1 is a measured step, not a finished model. The dataset flywheel, the evaluation harness, and the v2 recipe are the project; the checkpoint is just what the flywheel produced this week. Pair it with an agent harness like MenteE SWE and the model becomes the backend of a working terminal agent.
Author · MenteE AI — menteeai.org · syab.tech