Back to Xybrid
Live demo
The model runs here.
Your prompt. Your device. Try a small language model
on your GPU or CPU, right in this tab.
02 Response
Ready when you areA little intelligence.
Entirely on your side.
Your prompt stays in this tab.
No account. No API key. Inference happens here.
@xybrid/webBuild this into your appView the code
demo.ts
import { XybridLlm } from '@xybrid/web';
const llm = await XybridLlm.load(
'https://huggingface.co/xybrid-ai/SmolLM2-135M-Instruct-LiteRTLM/resolve/main/model_metadata.json',
{
wasmPath: '/xybrid/runtime', // the runtime files, served from your origin
accelerator: 'auto',
},
);
const stream = llm.generateStream(
"Write a haiku about AI that works offline.",
{ maxOutputTokens: 256 },
);
for await (const delta of stream) {
// Append each delta to your response.
console.log(delta);
} Updates with your prompt and engine choice.
Explore the web SDK ↗From this tab to your app.
Read the quickstart