MenteE AI · Open Research
Published research
Three open research lines, all with full numbers and honest limitations. Think While You Speak: a NumPy-only architecture where a memory-aware monitor decides when a model should think. MenteE Code 3B: a coding model built from 53,567 execution-verified agent trajectories, regression published on purpose. mentee-embed: trilingual embedding models (Arabic, English, Urdu) trained entirely from scratch, v1 through v4.
Think While You Speak: Adaptive Computational Effort in Neural Dialogue Systems
Shah, Syed Syab Ahmad · Team MenteE AI — MenteE AI
A minimal two-pathway architecture where a memory-aware monitor decides per input whether to think. Routing gradient: 2.8% of easy inputs deliberated, 99.7% of medium, 99.2% of hard. 93.5% accuracy at 67.2% deliberation (10 seeds), and a 3-seed ablation showing that removing the episodic novelty feature drops accuracy from 94.3% to 73.7%. Pure NumPy, CPU only.
Show BibTeX
@misc{shah2026acp,
title = {Think While You Speak: Adaptive Computational Effort
in Neural Dialogue Systems},
author = {Shah, Syed Syab Ahmad and Team MenteE AI},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.24353178},
url = {https://doi.org/10.5281/zenodo.24353178},
note = {MenteE AI, https://menteeai.org}
}Think While You Speak — Adaptive Cognitive Processing
The Architecture
Two pathways and a monitor. The fast path answers by default; a small episodic memory decides when the answer deserves a second look. Implemented in pure NumPy, CPU only.
Think While You Speak: Adaptive Computational Effort in Neural Dialogue Systems
Shah, Syed Syab Ahmad · Team MenteE AI — MenteE AI
A minimal two-pathway architecture where a memory-aware monitor decides per input whether to think. Routing gradient: 2.8% of easy inputs deliberated, 99.7% of medium, 99.2% of hard. 93.5% accuracy at 67.2% deliberation (10 seeds), and a 3-seed ablation showing that removing the episodic novelty feature drops accuracy from 94.3% to 73.7%. Pure NumPy, CPU only.
Show BibTeX
@misc{shah2026acp,
title = {Think While You Speak: Adaptive Computational Effort
in Neural Dialogue Systems},
author = {Shah, Syed Syab Ahmad and Team MenteE AI},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.24353178},
url = {https://doi.org/10.5281/zenodo.24353178},
note = {MenteE AI, https://menteeai.org}
}The four components
Fast pathway
A linear map, Wf·x + bf. Cheap by construction: a linear model cannot fit the quadratic or product-interaction targets, so the capability gap between the two paths is real rather than assumed.
Deliberate pathway
A 48-unit MLP over the concatenation [x, fast_out]. It can fit every target the fast path cannot, at roughly 54x the multiply-accumulates.
Monitor
p = σ(w·x + wn·novelty + b), a linear gate over the input plus one explicit episodic novelty scalar. The thresholded flag is the traceable routing decision.
Episodic memory
30 fixed keys (15 from the familiar cluster, 15 random). Novelty = 1 − max cosine similarity. Familiar inputs answer fast; novel inputs escalate.
Training objective
L = (y − y*)² + α · [−r log p − (1 − r) log(1 − p)] + λ · p · CdTask loss plus routed cross-entropy plus a compute penalty. The routing term teaches the monitor which inputs need the deliberate path; the penalty keeps always-deliberate from being free. Gradients are hand-derived and checked against finite differences.
Results
5,000 synthetic examples, three difficulty levels, 10 seeds. Task accuracy = |error| < 0.5. The hero summary below is generated straight from the evaluation JSON.
Main comparison — 10 seeds, mean ± std
| Model | Accuracy | Routing Acc. | Delib. Rate |
|---|---|---|---|
| ACP ★ | 0.935 ± 0.012 | 0.987 ± 0.002 | 0.672 ± 0.000 |
| Always Delib. | 0.937 ± 0.006 | 0.667 ± 0.000 | 1.000 |
| Fixed MLP | 0.947 ± 0.007 | 0.667 ± 0.000 | 1.000 |
| Random Router | 0.653 ± 0.043 | 0.501 ± 0.008 | 0.499 ± 0.005 |
| Always Fast | 0.370 ± 0.081 | 0.333 ± 0.000 | 0.000 |
Deliberation rate is the fraction of inputs routed to the expensive path. The ground-truth base rate is 66.7% (Medium + Hard).

Routing Gradient
Per-difficulty accuracy and deliberation. The point of the architecture is this column split: almost never deliberate on Easy, almost always on Medium and Hard.
Accuracy and deliberation by difficulty — mean across 10 seeds
| Model | Easy Acc | Med Acc | Hard Acc | Easy Delib | Med Delib | Hard Delib |
|---|---|---|---|---|---|---|
| ACP ★ | 0.977 | 0.965 | 0.863 | 0.028 | 0.997 | 0.992 |
| Always Fast | 0.298 | 0.549 | 0.264 | 0.000 | 0.000 | 0.000 |
| Always Delib. | 0.987 | 0.962 | 0.863 | 1.000 | 1.000 | 1.000 |
| Fixed MLP | 0.986 | 0.968 | 0.886 | 1.000 | 1.000 | 1.000 |
| Random Router | 0.638 | 0.759 | 0.562 | 0.496 | 0.503 | 0.498 |
The ground-truth policy deliberates on exactly 66.7% of inputs (Medium + Hard). ACP's 67.2% means it recovers the optimal policy almost exactly.
Memory Ablation
Retrain the model with the novelty term removed, so the monitor sees only the raw input. This isolates what the episodic memory contributes.
Routing ablation — 3 seeds, mean ± std
| Monitor | Accuracy | Routing Acc. | Delib. Rate |
|---|---|---|---|
| x + novelty | 0.943 ± 0.008 | 0.987 ± 0.001 | 0.672 ± 0.000 |
| x only | 0.737 ± 0.088 | 0.921 ± 0.003 | 0.662 ± 0.008 |
Without the novelty feature the monitor still deliberates on 66.2% of inputs overall, but sends 11.2% of Easy inputs and only 87.9% of Hard inputs to the deliberate path.

The memory is not decorative
The x-only monitor routes at a similar overall rate but misallocates deliberation, and the misroutes land exactly where they cost the most: 12% of Hard inputs go to a linear path that cannot fit them. That is a 20.6-point accuracy collapse from one removed feature.
Strengths
The routing gradient is near-optimal
Deliberation rises from 2.8% on easy inputs to 99.7% / 99.2% on medium and hard, matching the 66.7% base rate almost exactly, at 98.7% routing accuracy.
The capacity gap is real
The linear fast path provably cannot fit the quadratic (Medium) or product-interaction (Hard) targets, so the routing measurement is meaningful rather than a memorization artifact.
Memory does measurable work
Removing the novelty feature drops accuracy from 94.3% to 73.7%: the x-only gate deliberates at the same overall rate but sends 12% of hard inputs to the path that cannot fit them.
Reproducible without a GPU
Pure NumPy on CPU, 10 seeds, and hand-derived gradients verified against finite differences (max relative error 1.7e-9).
Limitations
Stated in the preprint itself. These are the boundaries of the claim.
Mechanism study, not language
The task is synthetic 16-dimensional regression. Dialogue is the motivating application, not the experimental setting.
Routing is supervised
The monitor trains on ground-truth route labels. A deployed system would not have those; label-free routing is the obvious next step.
The monitor is not free
The 30-key memory scan costs ~496 MACs per input vs ~864 for the deliberate path, so at 67% deliberation there is no net FLOP saving yet. A few keys or a low-dimensional sketch would move the crossover.
Novelty detector, not episodic memory
Keys are fixed and hand-filled, and the readout is a single scalar. Learned writes and content-based retrieval are future work.
Roadmap
Ordered by how much each step strengthens the central claim.
Label-free routing
Train the monitor from task loss plus a compute penalty, or from the fast path's own residual, so the system decides when to think without an answer key.
Content-addressable memory
Store key-value episodes and feed a readout vector into the monitor, then test on tasks where the same input needs different compute depending on stored context.
A monitor worth paying for
Shrink the memory to a handful of keys or a low-dimensional sketch so the routing policy turns into a real FLOP saving.
Language
Move from synthetic vectors to a small language task: streaming dialogue where deliberation is visible between turns.
Reproduction
Pure NumPy on CPU. No GPU, no training script longer than a hundred lines.
Full pipeline
pip install -r requirements.txt python test_gradients.py python evaluate.py python ablation_novelty.py python charts/make_results_table.py python charts/generate_charts.py
test_gradients.py verifies every hand-derived gradient against finite differences before anything else runs. Every table on this page is generated from evaluation_results.json and ablation_novelty.json.
MenteE AI · Open research · mentee-embed: trilingual embeddings trained from scratch (Arabic · English · Urdu) · Think While You Speak: adaptive computational effort, NumPy-only · MenteE Code 3B: a coding model built from execution-verified agent trajectories · Every table on this page is generated from the evaluation JSON in the public repositories.