Models
Right answers, not just fast ones
Simo-1 Pro on two decision benchmarks, with other models alongside. Simo answers each question directly, with no reasoning text written first.
DecideBench: 96.75%
DecideBench uses contrastive pairs: two near-identical situations that need opposite answers.
| Measure | Simo-1 Pro |
|---|---|
| Overall | 96.75% |
| Both halves of a pair right | 93.5% |
| Easy decisions | 98.26% |
| Hard decisions | 94.71% |
Hard decisions, against other models
| Model | Hard decisions |
|---|---|
| Simo-1 Pro (DecideBench hard) | 94.71 |
| Leading reasoning model | 91.7 |
| OneJev | 73.1 |
| Jev 1.13 | 65.3 |
| Jev-Omni | 62.5 |
Simo-1 Pro is scored on the hard split of DecideBench. OneJev, Jev 1.13, Jev-Omni and the reasoning model (run in thinking mode) are scored on the hard split of DecisionBench, a separate decision benchmark; the two benchmarks are not the same test.
Banking77: 81.8%
Banking77 asks a model to route a customer message to the right intent.
| Measure | Simo-1 Pro |
|---|---|
| Accuracy | 81.8% |
| Top-3 accuracy | 94.7% |
| Macro-F1 | 0.813 |
| Calibration error | 0.047 |
Scored on a random sample of the test split. Calibration error is the expected calibration error of the Choice probabilities.
How these were scored
Each row is one Choice question over the benchmark’s own options. The probabilities are what Simo returns directly; there is no reasoning text before the answer.
Frequently asked questions
How accurate is Simo?
Simo-1 Pro scores 96.75% on DecideBench and 81.8% on Banking77, with a top-3 accuracy of 94.7%.
How does Simo compare with a reasoning model?
On hard decisions Simo-1 Pro scores 94.71 on DecideBench hard, next to 91.7 for a leading reasoning model on DecisionBench hard. These are separate benchmarks. Simo also answers in milliseconds, with no reasoning text.