Introducing mentee-embed-v1: Training Competitive Multilingual Embeddings from Scratch for Arabic, English and Urdu

MenteE AI is building menteeai.org as an AI product company — not an agency. Our first public research artifact is mentee-embed-v1, a 41M-parameter trilingual text embedding model for Arabic, English and Urdu trained entirely from scratch — no pretrained backbone, no fine-tuning of giants. The preprint is published and citable at doi:10.5281/zenodo.22087139 (Zenodo, CC BY 4.0, v1.0.0).

What is mentee-embed-v1?

mentee-embed-v1 is a compact Transformer encoder (12 layers, 384 hidden, 41M params, 128 tokens) with a custom 50K ByteLevel BPE tokenizer trained on our own trilingual corpus. It outputs 384-dimensional L2-normalized vectors via mean pooling — ready for retrieval, RAG, semantic search and deduplication. Find the model card at /embed-models and the full technical report at /research.

How we trained it: MLM then relational distillation

Training competitive embeddings from random initialization is hard — small models collapse when given only sparse triplet labels. We solved it with a two-stage recipe: Stage A — Masked Language Modeling on 1.1M trilingual sentences (6,000 steps, public loss curve on the report), then Stage B — Relational Knowledge Distillation from intfloat/multilingual-e5-base. Instead of 1 bit per triplet, the student sees 960 dense numbers per batch (MSE on cosine similarity matrices + InfoNCE). That is how a 41M student learns the teacher's geometry. Code is at github.com/MenteE-s/mentee-embeddings.

Honest benchmarks: strengths and where we lag

We report two protocols fully. Protocol A (in-batch, ~97 candidates): mentee-embed-v1 reaches avg MRR@10 0.585 beating all-MiniLM-L6-v2 (0.449) and val acc@1 0.820 even above paraphrase-MiniLM-L12-v2 (0.795), plus cross-lingual EN↔UR 0.757 with no shared script. Protocol B (corpus-pool, 15K docs/lang): MRR@10 ~0.19 per language and R@100 ~0.47 — usable as a re-ranker, not a billion-document standalone engine — while mpnet-base hits 0.94/0.68/0.58. We publish both wins and limits. See charts on /research.

How to use it

import torch
from huggingface_hub import hf_hub_download
from src.model import build_embedder

model_pt = hf_hub_download("MenteEAI/mentee-embed-v1", "model.pt")
tok_path = hf_hub_download("MenteEAI/mentee-embed-v1", "tokenizer.json")

payload = torch.load(model_pt, map_location="cpu", weights_only=False)
model = build_embedder(payload["encoder_config"], payload["vocab_size"])
model.load_state_dict(payload["state_dict"])

Today weights load via src/model.py in the repo; Sentence-Transformers export is planned. Hugging Face: huggingface.co/MenteEAI/mentee-embed-v1.

Why open?

At MenteE AI we ship products, not decks. Publishing mentee-embed-v1 preprint, code, weights and even failures is our way to earn trust like DeepSeek, BGE and sentence-transformers — not hide like closed APIs. If you use it, please cite:

@software{shah2026menteeembed,
  title = {mentee-embed: Training Competitive
    Multilingual Text Embeddings from Scratch
    for Arabic, English, and Urdu},
  author = {Shah, Syed Syab Ahmad and
    Sania, Shakeel and Hamza, Rustam and
    Mahboob, Khan},
  year = {2026},
  publisher = {Zenodo},
  version = {v1.0.0},
  doi = {10.5281/zenodo.22087139},
  url = {https://doi.org/10.5281/zenodo.22087139}
}

What's next for MenteE and mentee-embed

Roadmap: v1.5 on the successful MLM foundation, public leaderboard vs MiniLM/E5/BGE, Apache 2.0 release, MTEB evaluation and Urdu-focused alignment — all tracked in the report. Follow menteeai.org, syab.tech and the MenteE Blog for updates.