---
title: "model_metadata.json: A Declarative Schema for ML Model Execution - Xybrid Blog"
description: "How a single JSON file replaces custom inference scripts and makes any ML model self-describing. No more 'works on my machine.'"
url: "https://www.xybrid.ai/blog/model-metadata-json-declarative-schema"
---

# model_metadata.json: A Declarative Schema for ML Model Execution

Engineering · July 10, 2026 · Glenn Sonna · 6 min read

How a single JSON file replaces custom inference scripts and makes any ML model self-describing. No more 'works on my machine.'

Every ML model you've ever deployed had the same hidden problem: **the inference script**.

That Python file with hardcoded tensor shapes, tokenizer paths, magic normalization constants, and platform-specific workarounds. The one that "just works" on the original developer's machine and breaks everywhere else.

We replaced it with a JSON file.

---

## The Problem

Here's what typical model deployment looks like:

```python
# inference.py — "just copy this file with the model"
import onnxruntime as ort
import numpy as np
from tokenizer import load_tokenizer  # custom module, good luck finding it

session = ort.InferenceSession("model.onnx")
tokenizer = load_tokenizer("vocab.json")

def run(text):
    tokens = tokenizer.encode(text)
    tokens = np.array([tokens], dtype=np.int64)
    # Magic: pad to 128 because the model was exported that way
    if tokens.shape[1] < 128:
        tokens = np.pad(tokens, ((0,0), (0, 128 - tokens.shape[1])))
    outputs = session.run(None, {"input_ids": tokens})
    # Magic: take argmax of last dim, skip first token, decode
    predicted = np.argmax(outputs[0], axis=-1)[0][1:]
    return tokenizer.decode(predicted)
```

This script contains critical deployment knowledge:
- Input tensor name is `input_ids`
- Input must be `int64`
- Padding to 128 is required
- Output needs argmax → skip first → decode

None of this is discoverable from the ONNX file alone. Lose the script, lose the ability to run the model.

## The Solution: model_metadata.json

Every model in Xybrid ships with a `model_metadata.json` that **declares everything needed to run it**:

```json
{
  "model_id": "wav2vec2-base-960h",
  "version": "1.0",
  "description": "Speech recognition using CTC decoding",

  "execution_template": {
    "type": "SimpleMode",
    "model_file": "model.onnx"
  },

  "preprocessing": [
    {
      "type": "AudioDecode",
      "sample_rate": 16000,
      "channels": 1
    }
  ],

  "postprocessing": [
    {
      "type": "CTCDecode",
      "vocab_file": "vocab.json",
      "blank_index": 0
    }
  ],

  "files": ["model.onnx", "vocab.json"],

  "metadata": {
    "task": "speech-recognition",
    "architecture": "wav2vec2",
    "language": "en"
  }
}
```

The runtime reads this file, builds the execution pipeline, and runs it. No inference script. No custom code. No implicit knowledge.

## How It Works

The `TemplateExecutor` (Xybrid's core execution engine) processes the metadata in three phases:

```text
Input Envelope
      │
      ▼
┌─────────────┐
│ Preprocess   │  AudioDecode: WAV bytes → f32 samples at 16kHz
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ Model        │  ONNX inference: samples → logits
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ Postprocess  │  CTCDecode: logits → text via vocab.json
└──────┬──────┘
       │
       ▼
Output Envelope
```

In Rust:

```rust
let metadata: ModelMetadata = serde_json::from_str(
    &std::fs::read_to_string("model_metadata.json")?
)?;

let mut executor = TemplateExecutor::with_base_path("./model-dir");
let output = executor.execute(&metadata, &input)?;
```

Three lines. Works for ASR, TTS, classification, embeddings — any model type. The metadata file is the only thing that changes.

## The Schema

### Execution Templates

The `execution_template` field defines how the model file is loaded and run:

```jsonc
// Standard ONNX model
{ "type": "SimpleMode", "model_file": "model.onnx" }

// GGUF model (llama.cpp)
{
  "type": "Gguf",
  "model_file": "model.gguf",
  "context_length": 4096
}

// Whisper (Candle, autoregressive)
{
  "type": "WhisperMode",
  "model_file": "model.safetensors",
  "tokenizer_file": "tokenizer.json",
  "config_file": "config.json"
}
```

The template type tells the executor which backend to use (ONNX Runtime, llama.cpp, Candle) and what files are needed.

### Preprocessing Steps

Available preprocessing steps and what they do:

| Step | Input | Output | Config |
|------|-------|--------|--------|
| `AudioDecode` | WAV bytes | f32 samples | `sample_rate`, `channels` |
| `Phonemize` | Text | Token IDs | `backend`, `tokens_file`, `dict_file` |
| `Tokenize` | Text | Token IDs | `vocab_file`, `max_length` |

Example — TTS preprocessing:

```json
{
  "preprocessing": [
    {
      "type": "Phonemize",
      "backend": "MisakiDictionary",
      "tokens_file": "tokens.txt"
    }
  ]
}
```

This converts `"Hello world"` → `[phoneme token IDs]` using a built-in phonemizer. No external dependencies (no espeak-ng installation needed).

### Postprocessing Steps

| Step | Input | Output | Config |
|------|-------|--------|--------|
| `CTCDecode` | Logits tensor | Text | `vocab_file`, `blank_index` |
| `TTSAudioEncode` | f32 waveform | WAV bytes | `sample_rate`, `apply_postprocessing` |
| `ArgMax` | Logits tensor | Class index | — |

### Voice Configuration (TTS-Specific)

TTS models can declare available voices:

```json
{
  "metadata": {
    "voices": {
      "voices_file": "voices.bin",
      "format": "Float32Array",
      "embedding_dim": 256,
      "voices": [
        { "id": "af_heart", "name": "Heart", "language": "en-us" },
        { "id": "af_sky", "name": "Sky", "language": "en-us" }
      ],
      "default_voice": "af_heart"
    }
  }
}
```

The runtime loads the voice embedding from `voices.bin` and passes it as an additional model input. Consumers just pick a voice by name.

## Why Not ONNX Model Zoo Format? Or MLflow?

**ONNX Model Zoo** has model cards, but they're documentation, not execution configuration. They don't tell you how to preprocess inputs or postprocess outputs.

**MLflow** has model signatures, but they describe tensor shapes, not the pipeline around the model. You still need custom code for tokenization, audio decoding, etc.

**model_metadata.json** is execution-complete: everything from "raw user input" to "usable output" is declared. The runtime needs zero custom code per model.

| | ONNX Zoo | MLflow | model_metadata.json |
|-|----------|--------|-------------------|
| Tensor shapes | Yes | Yes | Implicit |
| Preprocessing | No | No | Yes |
| Postprocessing | No | No | Yes |
| Multi-file models | No | Partial | Yes (`files` array) |
| Voice/variant config | No | No | Yes |
| Runtime-executable | No | Partial | Yes |

## Real Examples

### ASR (wav2vec2)

```json
{
  "model_id": "wav2vec2-base-960h",
  "execution_template": { "type": "SimpleMode", "model_file": "model.onnx" },
  "preprocessing": [
    { "type": "AudioDecode", "sample_rate": 16000, "channels": 1 }
  ],
  "postprocessing": [
    { "type": "CTCDecode", "vocab_file": "vocab.json", "blank_index": 0 }
  ]
}
```

Input: WAV audio bytes → Output: transcribed text.

### TTS (Kokoro)

```json
{
  "model_id": "kokoro-82m",
  "execution_template": { "type": "SimpleMode", "model_file": "model.onnx" },
  "preprocessing": [
    { "type": "Phonemize", "backend": "MisakiDictionary", "tokens_file": "tokens.txt" }
  ],
  "postprocessing": [
    { "type": "TTSAudioEncode", "sample_rate": 24000, "apply_postprocessing": true }
  ]
}
```

Input: text string → Output: WAV audio bytes.

### LLM (Qwen 3.5)

```json
{
  "model_id": "qwen3.5-0.8b",
  "execution_template": {
    "type": "Gguf",
    "model_file": "Qwen3.5-0.8B-Q4_K_M.gguf",
    "context_length": 4096
  },
  "preprocessing": [],
  "postprocessing": [],
  "metadata": {
    "task": "text-generation",
    "architecture": "qwen35",
    "backend": "llamacpp"
  }
}
```

Input: text prompt → Output: generated text. No pre/postprocessing needed — the GGUF backend handles tokenization internally.

## Extending the Schema

Need a new preprocessing step? Add a variant to the enum and implement the handler:

```rust
// In steps.rs
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type")]
pub enum PreprocessingStep {
    AudioDecode { sample_rate: u32, channels: u16 },
    Phonemize { backend: PhonemizerBackend, tokens_file: String },
    Tokenize { vocab_file: String },
    // Add your new step:
    MelSpectrogram { n_fft: usize, hop_length: usize, n_mels: usize },
}
```

Implement the handler, and every model that uses `MelSpectrogram` in its metadata will automatically work across all platforms.

## The Payoff

With `model_metadata.json`:

- **Model authors** declare how their model runs, once
- **Runtime developers** implement preprocessing/postprocessing steps, once
- **App developers** write `executor.execute(&metadata, &input)` — no model-specific code
- **New platforms** (Flutter, Swift, Kotlin) get every model for free — they use the same metadata

One JSON file. Zero inference scripts. Every model is self-describing and portable.

---

**See it in action**: [github.com/xybrid-ai/xybrid](https://github.com/xybrid-ai/xybrid) — browse `fixtures/models/` for real model_metadata.json examples.

---

*Thoughts on the schema design? Things we should add? Let us know in the comments.*

---

Tags: machinelearning, architecture, opensource, ai

## Keep reading

- [Publishing ML Models as Bundles with xybrid-pack](https://www.xybrid.ai/blog/publishing-ml-models-as-bundles-xybrid-pack) — A packaging pipeline that turns raw model files into versioned, checksummed, cross-platform bundles hosted on HuggingFace. No more 'download the model and put it here.'
- [Building a Cross-Platform ML Inference SDK in Rust](https://www.xybrid.ai/blog/building-cross-platform-ml-inference-sdk-in-rust) — How we built a single Rust core that powers ML inference across CLI, Flutter, Swift, Kotlin, and Unity — and the architectural decisions that made it possible.
- [Vendoring llama.cpp in a Rust Workspace (Lessons Learned)](https://www.xybrid.ai/blog/vendoring-llamacpp-in-rust-workspace) — The practical pain of embedding a C++ inference engine in a Rust monorepo — cross-compilation, Android fp16, think-tag stripping, and when to give up on upstream.
