On-device AI leaderboard

Inference engines and models, ranked on the phones and laptops people own. Every rank links to the experiments and raw data behind it.

Planned Not ranked yet

Which speech models keep up with a conversation on the device?

Speech recognition and text-to-speech, streamed as a voice app would. Filter by engine, or let each model run on its best compatible engine.

Each row
Speech model configurations: recognition and synthesis, on an engine. By default each model runs on its best compatible engine for the objective; filter by engine to fix it.
Views
Overall · Desktop · Mobile
Filters
Engine · Hardware · Workload · Quantization
Measures
Word error rate · Real-time factor · Latency to first audio · Peak memory

Rank by

  • BalancedAccuracy first, then real-time factor and memory.
  • SpeedLatency to the first word or the first audio, and real-time factor.
  • MemoryPeak memory while streaming.

Its methodology comes first. Nothing is ranked here until its experiments are published with their raw data.